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ABSTRACT 


A  prototype  interface  was  developed  to  support  military  practitioners  with 
enhanced  levels  of  situation  awareness  and  better  decision  making  as  they  conduct 
command  and  control  activities  during  tactical  operations.  A  laboratory  experiment  was 
conducted  to  evaluate  the  capability  of  this  interface’s  cognitive  systems  engineering  and 
ecological  interface  design  principles  to  support  critical  activities  (i.e.,  assess  anticipated 
enemy  actions  on  friendly  force  operations).  Qualitative  tactical  simulations  and  an 
alternative  interface  (an  experimental  version  of  an  existing  U.S.  Army  interface)  were 
developed.  Participants  were  blocked  against  one  interface  and  provided  estimates  of 
perceived  levels  of  cognitive  workload  while  collecting,  integrating,  and  reporting 
various  forms  of  friendly  and  enemy  force  infonnation  during  two  realistic  tactical 
scenarios.  The  results  suggested  that  the  prototype  interface  produced  significantly  better 
performance  in  six  out  of  seven  statistical  comparisons  examined.  The  cognitive  systems 
engineering  and  ecological  interface  design  strategy  was  very  effective  in  this 
experimental  context.  The  potential  for  this  design  to  be  useful  for  other  complex  work 
domains  is  explored.  Actual  or  potential  applications  of  this  study  include  both  specific 
interface  design  strategies  for  military  command  and  control  and  general  interface  design 
principles  for  civil  transportation  work  domains. 
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EXECUTIVE  SUMMARY 


Information  is  vital  for  success  during  war.  Since  the  turn  of  the  twentieth 
century,  information  technology  has  continued  to  advance  at  an  exponential  rate, 
enabling  commanders  to  take  advantage  of  the  speed,  range,  and  lethality  of  modern 
weaponry.  Information  technology  advancements  also  make  warfare  more  complex. 
Since  the  inception  of  the  Force  XXI  modernization  program,  the  U.S.  Anny  has  fielded 
numerous  interfaces  to  leverage  the  capabilities  of  digital  infonnation  systems  during  the 
command  and  control  of  tactical  operations.  In  reality,  many  of  these  information 
technologies  have  given  little  consideration  to  the  role  of  the  human  in  the  design  or 
implementation  of  the  systems.  This  failure  to  recognize  humans  as  the  critical  nodes 
within  information  system  designs  ultimately  degrade  total  system  performance  and 
further  complicate  command. 

Ultimately,  success  in  enhancing  decision-maker  performance  relies  on  the 
integration  of  human  cognition  and  technological  capability.  This  reliance  on  both 
human  and  machine  agents  strongly  suggests  a  Human  Systems  Integration  approach, 
which  strives  to  implement  people  as  the  key  elements  within  the  “system  of  systems” 
architecture  by  assisting  in  system  designs  that  support  human  limitations  and  enhance 
human  strengths. 

Information  complexity  in  itself  is  not  a  problem,  given  meaningful  information  is 
presented  in  a  coherent  and  structured  manner.  The  essential  notion  being  that  in  order 
for  information  system  technologies  to  improve  total  system  performance  and 
effectiveness,  information  must  be  constructed  into  representations  that  exploit  the 
inherent  pattern-recognition  capabilities  of  the  human,  while  also  decreasing  reliance  on 
limited-capacity  resources.  Accordingly,  this  study  explores  the  effectiveness  of  a 
prototype  interface  (RAPTOR)  designed  to  be  used  in  command  and  control  during 
tactical  operations.  The  extent  to  which  warfighter  performance  is  enhanced  by  the 
cognitive  systems  engineering  approach  and  ecological  interface  design  principles  used  to 
develop  RAPTOR  are  made  explicit. 
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During  this  study,  a  laboratory  experiment  was  conducted  to  evaluate  the 
capability  of  the  interface  to  support  military  practitioners  as  they  conducted  critical 
command  and  control  activities  (i.e.,  assess  anticipated  enemy  actions  on  friendly  force 
operations)  during  tactical  operations.  U.S.  Army  officers  with  previous  combat 
experience  in  Operation  Iraqi  Freedom  and/or  Operation  Enduring  Freedom  served  as 
participants.  Two  qualitative  tactical  simulations  and  an  alternative  interface  (an 
experimental  version  of  an  existing  U.S.  Army  interface)  were  developed.  Participants 
were  blocked  against  one  interface  and  provided  estimates  of  perceived  cognitive 
workload  while  collecting,  integrating,  and  reporting  various  forms  of  friendly  and 
enemy  force  information  during  the  tactical  simulations. 

The  results  suggested  that  the  prototype  interface  produced  significantly  better 
performance  in  six  out  of  seven  statistical  comparisons  examined.  The  cognitive  systems 
engineering  and  ecological  interface  design  strategy  was  very  effective  in  this 
experimental  context.  The  results  also  demonstrate  the  potential  for  this  design  strategy 
to  be  useful  for  other  complex  work  domains.  Actual  or  potential  applications  of  this 
study  include  both  specific  interface  design  strategies  for  military  command  and  control 
and  general  interface  design  principles  for  civil  transportation  work  domains. 

In  conclusion,  researchers  believe  results  from  this  study  achieved  the  primary 
purpose  of  assisting  the  U.S.  Anny  in  its  efforts  to  develop  advanced  C  interfaces  that 
account  for  human  capabilities  and  limitations. 
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I.  INTRODUCTION 


Information  is  vital  for  success  during  war.  According  to  Shapiro  (1999),  this 
truism  dates  back  at  least  to  the  ancient  Chinese  writings  of  Sun  Tzu.  Accurate 
information  on  friendly  and  enemy  activities  reduces  what  Clausewitz  called  the  “fog  of 
war”  and  facilitates  the  rapid  defeat  of  enemy  forces  on  the  battlefield.  Since  the  turn  of 
the  twentieth  century,  information  technology  has  continued  to  advance  at  an  exponential 
rate,  enabling  commanders  to  take  advantage  of  the  speed,  range,  and  lethality  of  modern 
weaponry.  Bennett,  Posey,  and  Shattuck  (2008)  suggest  military  applications  continue  to 
provide  both  the  impetus  for  technological  change  and  a  testing  ground  for 
these  applications. 

Information  technology  advancements  also  make  warfare  more  complex.  Since 
the  inception  of  the  Force  XXI  modernization  program,  the  United  States  Army  has 
fielded  numerous  interfaces  to  leverage  the  capabilities  of  digital  information  systems 
(INFOSYS)  during  the  command  and  control  (C  )  of  tactical  operations.  Many  of  these 
interfaces  purport  to  enhance  situational  awareness  (SA)  by  allowing  the  commander  to 
access  countless  gigabytes  of  near  real-time  information.  However,  the  sheer  volume  of 
information  these  sophisticated  technologies  provide  often  overwhelms  the  commander’s 
ability  to  comprehend  their  meaning.  This  natural  phenomenon  of  information  overload 
degrades  SA  and  increases  the  fog  of  war.  As  Hall  (2000)  asserts,  many  more  years  of 
development  and  experimentation  are  required  before  computers  can  meet  all  the 
conditions  necessary  to  control  modern  forces  on  the  battlefield.  The  goal  of  this  thesis  is 
to  assist  the  U.S.  Army  in  its  efforts  to  develop  and  incorporate  interfaces  that  provide 
effective  support  to  military  practitioners  as  they  cope  with  the  inherent  complexities 
of  C2. 

A.  CHARACTERISTICS  OF  THE  C2  DOMAIN 

According  to  U.S.  Army  Field  Manual  (FM)  6.0  (2003),  through  command  and 
control,  the  commander  initiates  the  actions  of,  influences,  and  synchronizes  the  elements 
of  combat  power  to  impose  his  will  on  the  situation  and  defeat  the  enemy.  The  critical 
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role  C  plays  in  success  on  the  battlefield  is  not  a  new  concept.  From  the  ancient 
battlegrounds  of  Mesopotamia  to  the  asymmetric  combat  zones  of  modern-day 

Afghanistan  and  Iraq,  military  commanders  have  practiced  C  throughout  the  history  of 
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warfare.  While  the  size  and  scope  of  military  operations  has  transformed  the  theory  of  C 
over  the  years,  mission  accomplishment  has  continued  to  remain  its  goal. 

During  tactical  operations,  C  is  complex  and  dynamic.  The  fluid  nature  and 
harsh  conditions  of  combat  create  friction,  which  makes  performing  even  simple  tasks 
extremely  difficult  (Kemmerer,  2008).  Friction  stems  from  multiple  sources,  but  is 
routinely  characterized  by  time-pressure,  high  personal  stakes  (risk),  and  uncertainty 
(Lipshitz,  Klein,  Orasanu,  &  Salas,  2001).  Time -pressure  occurs  because  events  during 
tactical  operations  are  extremely  fast-paced.  It  is  impossible  to  eliminate  risk  from 
combat  operations  since  the  loss  of  life  is  always  a  possibility.  Uncertainty  is  inevitable 
due  to  the  ways  in  which  data  are  observed,  measured,  and  reported.  Further,  the  enemy 
is  actively  involved  in  deception  and  misdirection.  The  scope,  complexity,  and  severity 
of  the  challenges  within  this  domain  of  application  are  perhaps  unprecedented  (Bennett 
et  ah,  2008). 

The  complexity  and  uncertainty  of  modern  warfare  makes  it  impossible  for 
commanders  to  effectively  execute  C  in  isolation.  Commanders  require  support  from 
staff  personnel  to  coordinate  and  synchronize  finite  resources  such  as  people,  equipment, 
technology,  and  logistics  to  achieve  mission-related  goals.  Commanders  also  need  help 
to  determine  the  effects  numerous  interrelated  factors  (e.g.,  enemy  forces,  terrain, 
weather,  time,  etc.)  will  have  on  these  resources  as  they  execute  tasks.  At  every  echelon 
of  command,  each  commander  has  a  C"  system  to  provide  that  support  (Department  of  the 
Army,  2003). 

According  to  FM  6-0  (2003),  the  C  system  strives  to  reduce  uncertainty  to 
manageable  levels  by  enabling  commanders  to  see  themselves,  the  enemy,  and  the 
terrain.  C  is  part  of  the  Battlefield  Operating  System  (BOS),  and  integrates  functions 
from  the  other  six  BOSs  to  accomplish  the  mission.  The  other  BOSs  include: 

•  Intelligence  System 

•  Maneuver  System 
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•  Fire  Support  System 

•  Air  Defense  System 

•  Mobility/Counter-mobility/Survivability  System 

•  Combat  Service  Support  System 

C  consists  of  two  components:  the  commander  and  his  C  system.  Commanders 
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use  their  C  systems  to  exercise  C  over  their  forces  to  accomplish  missions  (Department 
of  the  Anny,  2003).  The  commander,  while  having  overall  responsibility  for  C  ,  must 
understand  the  situation  before  making  decisions.  Staff  personnel  assist  the  commander 
by  collecting  and  processing  data  into  relevant  information  (RI).  Through  RI,  the 
commander  gains  understanding.  The  commander  then  uses  his  understanding  to 
determine  plausible  courses  of  action  (COA),  and  to  issue  timely  directives. 

Due  to  friction,  uncertainty  will  always  exist  regardless  of  how  well  the  C  system 
operates.  Commanders  and  staffs  must  use  information-focused  and  action-focused 
solutions  to  identify  and  reduce  uncertainty  (Department  of  the  Army,  2003). 
Action-focused  solutions  consist  of  experience,  training,  and  standard  operating 
procedures  (SOP).  Information-focused  solutions  increasingly  rely  on  computer-assisted 
information  distribution,  integration,  and  display  processes.  Thus,  today’s  military 
practitioner  requires  an  advanced  C  interface  that  assists  in  seeing  and  understanding  the 
entire  battlefield. 

B.  THE  RAPTOR  INTERFACE 

RAPTOR  (Representation  Aiding  Portrayal  of  Tactical  Operations  Resources)  is  a 
prototype  interface  designed  to  be  used  in  C  during  tactical  operations  (see  Figure  1). 
This  study  will  examine  if  warfighter  perfonnance  is  enhanced  by  the  cognitive  systems 
engineering  (CSE)  approach  and  ecological  interface  design  (EID)  principles  used  to 
develop  RAPTOR.  The  intent  behind  the  CSE  approach  to  RAPTOR’S  design  is  to 
provide  the  user  with  robust  cognitive  support  to  increase  perfonnance  during  C  efforts. 
Thus,  the  major  goals  for  the  design  and  implementation  of  RAPTOR  are  to  facilitate 
better  decision  making  and  enhance  operator  SA  by  increasing  understanding  of 
the  battlespace. 
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Figure  1.  RAPTOR  Interface  (After:  Bennett,  Posey,  &  Shattuck,  2008). 

Ultimately,  the  goal  of  interface  design  must  be  considered  as  the  provision  of 
effective  decision  and  problem-solving  support  (Bennett  et  ah,  2008).  With  this  goal  in 
mind,  the  extent  to  which  an  interface  enhances  operator  SA  and  closes  the  “information 
gap”  will  determine  the  benefit  of  any  interface  to  a  tactical  C“  system  (Endsley  & 
Garland,  2000).  Therefore,  in  the  context  of  decision  making,  understanding,  and  SA,  the 
degree  to  which  the  interface  design  improves  human  and  machine  interactions  will 
determine  total  system  performance.  The  more  intuitive  the  system  is  to  operate,  the 
faster  users  can  effectively  incorporate  the  system  into  the  work  domain.  Accordingly, 
the  design  principles  and  techniques  used  to  develop  RAPTOR  will  ultimately  assist  the 
U.S.  Army  in  its  efforts  to  incorporate  advanced  interfaces  that  account  for  human 
capabilities  and  limitations  in  the  U.S.  Army’s  Battle  Command  System  (ABCS). 

A  controlled  laboratory  experiment  was  the  primary  vehicle  used  to  assess 

military  decision  makers’  performance  while  performing  critical  battlefield  activities 

(e.g.,  gain  and  analyze  critical  knowledge  on  the  effects  of  terrain;  assess  anticipated 

enemy  actions  on  friendly  force  operations).  Previous  empirical  examinations  comparing 

RAPTOR  to  C  interfaces  currently  being  utilized  by  the  United  States  Army  have 

already  demonstrated  RAPTOR’S  potential  to  provide  superior  support  to  military 
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practitioners  as  they  execute  C  activities.  This  study  advances  the  development  of 
RAPTOR  by  identifying  areas  for  improvement  in  the  interface  design.  This  study  also 
builds  on  work  previously  conducted,  and  further  validates  the  ability  of  the  interface  to 
increase  performance  as  users  deal  with  uncertainty  and  novel  situations  in  dynamic  and 
fluid  environments.  Though  the  current  version  of  RAPTOR  was  inspired  by  the 
complexities  in  C  ,  the  results  of  this  study  could  potentially  provide  warfighters  with  a 
robust  interface  capable  of  assisting  and  supporting  a  variety  of  complicated  tasks  across 
a  broad  range  of  complex  work  domains. 

In  summary,  the  goal  of  this  thesis  was  to  demonstrate  how  effectively  RAPTOR 
employed  CSE  and  EID  constructs  to  increase  the  military  practitioner’s  understanding  of 
the  battlespace  and  to  enhance  SA.  Though  no  interface  will  result  in  complete 
understanding  or  perfect  SA,  an  interface  designed  to  support  the  cognitive  capabilities 
and  limitations  of  the  commander  may  lead  to  a  significant  tactical  edge  over  threat 
forces  (Libicki  &  Johnson,  1996). 

C.  HUMAN  SYSTEMS  INTEGRATION  (HSI) 

This  study  explores  the  effectiveness  of  RAPTOR  in  the  C  system  from  the 
perspective  of  HSI.  According  to  the  principles  of  HSI,  human  considerations  must  be 
viewed  as  a  priority  in  systems  design  to  reduce  life-cycle  costs  and  improve  total  system 
performance  (Miller  &  Shattuck,  2008).  HSI  strives  to  implement  the  human  as  the  key 
element  in  the  “system  of  systems”  architecture  by  assisting  in  system  designs  that 
support  human  limitations  and  enhance  human  strengths.  In  order  to  accomplish  this 
goal,  HSI  practitioners  focus  on  inherent  trade-offs  within  each  of  the  following 
HSI  domains: 

•  Human  Factors  Engineering 

•  Manpower 

•  Personnel 

•  Training 

•  Human  Survivability 

•  Health  Hazards 
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•  System  Safety 

•  Habitability 

Considerations  of  human-centered  design  trade-offs  early  in  the  acquisition 
process  promote  system  effectiveness,  safety,  and  cost  savings  throughout  the  system  life 
cycle  (Miller  &  Shattuck,  2008).  Further  explanation  of  the  domains,  and  the  specific 
trade-offs  within  each  domain,  can  be  found  in  Department  of  Defense  Instruction 
5000.2,  or  Handbook  of  Human  Systems  Integration  (Booher,  2003). 

For  the  purpose  of  this  study,  exploration  on  RAPTOR’S  potential  to  assist  and 
reinforce  the  military  commander’s  ability  to  actively  execute  all  aspects  of  C2will  focus 
on  the  HSI  domains  of  human  factors  engineering,  training,  and  manpower.  This  study 
hopes  to  highlight  the  extent  to  which  RAPTOR  improves  human  perfonnance,  and  to 
determine  how  well  the  interface  and  the  human  interact.  Essentially,  the  level  of 
achieved  human-computer  interaction  will  drive  the  time,  cost,  and  types  of  training 
required  for  users  to  successfully  operate  the  interface.  The  more  intuitive  RAPTOR  is  to 
operate,  the  faster  users  can  effectively  incorporate  the  interface  into  the  C  system.  The 
results  could  potentially  reduce  the  manpower  required  to  assist  commanders  at 
conducting  C  . 

D.  RESEARCH  QUESTIONS 

The  role  of  sophisticated  computer  technology  in  the  C  system  raises  several 
important  questions.  The  following  questions  set  this  study’s  foundation,  and  also 
provide  a  basis  for  future  studies  aimed  at  designing  interfaces  capable  of  assisting  and 
supporting  a  variety  of  complicated  tasks  across  a  broad  range  of  complex  work  domains. 
When  researching  the  problem  from  an  HSI  perspective,  one  must  focus  on  symbiotic 
relationships  that  exist  between  the  human  and  the  machine. 

•  To  what  extent  does  sophisticated  INFOSYS  technology  facilitate  C  ?  To 
what  extent  does  sophisticated  technology  impede  C2? 

•  How  do  humans  transform  data  into  infonnation?  How  do  these  processes 
enable  military  commanders  to  understand  the  battlespace? 


6 


•  How  do  interfaces  assist  humans  with  the  cognitive  integration  of 
information? 

•  How  are  perception  and  SA  intertwined?  How,  at  all,  does  SA  affect 
decision  making? 

•  How  does  an  interface  design  affect  the  man  and  machine  symbiosis? 

2 

•  To  what  extent  does  RAPTOR  increase  warfighter  perfonnance  during  C 
of  tactical  operations? 

E.  RESEARCH  OBJECTIVES 

The  ultimate  objective  of  this  thesis  is  to  facilitate  interface  designs  that  enhance 
total  system  performance.  The  specific  objectives  for  the  study  are: 

•  Develop  a  methodology  to  assess  how  interfaces  assist  humans  to  achieve 
enhanced  levels  of  SA  and  improved  decision  making. 

•  Develop  tactical  scenarios  for  modeling  and  simulation. 

•  Determine  the  extent  to  which  warfighter  performance  is  enhanced  by 
RAPTOR’S  CSE  and  EID  design  approach. 

•  Emphasize  the  importance  of  HSI  in  exploring  the  role  of  sophisticated 
computer  technologies  in  the  C  system. 

•  Provide  a  clear  direction  for  future  studies  aimed  at  incorporating 
advanced  interfaces  into  the  U.S.  Army’s  C  system. 

F.  THESIS  ORGANIZATION 

A  traditional  format  will  guide  the  organization  of  this  thesis.  Chapter  II  provides 
a  detailed  examination  of  the  CSE  framework,  and  EID  theoretical  constructs  used  to 
guide  RAPTOR’S  interface  design.  The  process  of  transforming  information  into 
individual  and  shared  understanding  is  further  described.  Also,  the  potential  for  interface 
designs  to  enhance  military  practitioner  SA  and  decision-making  are  made  explicit. 

Chapter  III  focuses  on  the  empirical  evaluation  of  the  RAPTOR  interface. 
Detailed  descriptions  of  the  methodology  (e.g.,  tactical  scenarios,  simulations,  data 
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collection  techniques)  used  during  experimental  events  are  provided.  Additionally, 
lessons  learned  during  the  experimental  events  are  documented  to  assist  researchers  with 
future  replications  of  the  study. 

Chapter  IV  reports  the  analysis  of  data  collected  during  experimental  events, 
while  Chapter  V  discusses  the  implications  of  the  data.  Chapter  VI  provides  conclusions 
and  recommendations,  and  illustrates  a  “way  ahead”  for  the  development  and  evaluation 
of  future  interfaces  designed  for  incorporation  into  complex  work  domains. 

Appendixes  are  included  to  provide  more  detailed  information  about  the 
experimental  process.  Appendix  A  and  B  provide  copies  of  the  tactical  scenario 
operation  orders  that  participants  reviewed  to  prepare  themselves  for  the  experimental 
trials.  Appendix  C  is  the  demographic  survey  used  to  compile  participant  professional 
data.  Appendix  D  and  E  are  the  post-training  tests  used  to  ensure  participants  retained 
the  comprehensive  knowledge  necessary  to  advance  to  the  experimental  trials.  Appendix 
E  is  the  feedback  survey  used  to  elicit  specific  comments  from  participants  once  they 
concluded  all  experimental  trails.  Appendix  G  illustrates  the  ad-hoc  notes  and  tables 
Baseline  interface  participants  created  during  the  experimental  trials. 
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II.  LITERATURE  REVIEW 


A.  C2  DURING  INFORMATION  AGE  WARFARE 

1.  The  Role  of  Information  Systems  Technology  in  the  C2  System 

According  to  FM  6-0  (2003),  the  object  of  INFOSYS  technology  is  to  enhance  the 
performance  of  people.  Twentieth  century  warfare  was  fought  primarily  through 
platform-centric  operations  (PCO)  (Department  of  Defense,  2001).  Radios  and 
telephones  were  the  predominate  means  of  communication  between  the  commander  and 
his  subordinate  leaders.  Procedures  for  receiving  and  disseminating  critical  information 
proved  extremely  cumbersome,  which  often  created  confusion  and  hindered  execution. 
As  a  result,  commanders  tended  to  make  centralized  decisions  and  commanded  by  plan, 
which  ultimately  depended  on  highly  trained  and  disciplined  Soldiers  to  carry  out  the 
plan  as  ordered  (Phister,  Busch,  &  Plonisch,  2003). 

Operation  Desert  Storm  exhibited  the  advantages  information  age  technologies 
can  provide  to  military  commanders  during  combat.  Lessons  learned  from  the 
Iraqi  Army’s  resounding  defeat  precipitated  the  U.S.  Army’s  modernization  process  to 
“digitize”  their  entire  INFOSYS  structure.  Since  1991,  the  evolution  of  INFOSYS 
technologies  (e.g.,  satellite  and  digital  communications)  has  ushered  in  network-centric 
operations  (NCO)  as  a  new  theory  of  warfare  (Department  of  Defense,  2001).  NCO’s 
concept  is  based  on  the  use  of  technology  to  increase  military  effectiveness  by  enabling 
warfighters  to  rapidly  share  and  utilize  battlefield  information  during  tactical  operations. 
Simply  put,  realizing  the  full  potential  that  advancements  in  computer,  sensor,  and 
communications  technology  can  have  on  reducing  the  friction  of  war  is  sine  qua  non  for 
NCO  (Libicki  &  Johnson,  1996). 

However,  due  to  the  speed  and  nonstop  tempo  of  the  modern  battlefield,  no  C 
system  can  work  without  INFOSYS  capable  of  leveraging  information-age  technologies 
(Department  of  the  Army,  2003).  As  a  result,  numerous  automated  command,  control, 
communications,  computer  applications,  and  intelligence  (C4I)  interfaces  have  been 
incorporated  into  the  ABCS.  Essentially,  ABCS  integrates  BOS  INFOSYS  to  share  near 
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real-time  information  vertically  and  horizontally  through  strategic,  operational,  and 
tactical  commands.  Force  XXI  Battle  Command,  Brigade  and  Below  (FBCB2)  is  the 
Army’s  primary  C  interface  integrated  into  the  ABCS.  FBCB2  displays  graphical 
representations  of  tactical  information,  and  can  be  used  either  on  the  move  or  in  fixed 
command  posts  (CP).  This  robust  network  of  integrated  INFOSYS  facilitates  the 
commander’s  ability  to  act  faster  than  the  enemy,  rather  than  just  reacting  to 
enemy  actions. 

Commanders  and  staffs  who  conduct  C  from  “digital”  tactical  operation  centers 
(TOC)  have  the  capability  to  receive,  process,  share,  disseminate,  and  display 
information  must  faster  than  those  who  conduct  C  from  “analog”  TOCs.  Computers 
perform  many  lower  order  functions  faster  and  more  efficiently  than  humans.  When  used 
correctly,  the  speed  and  efficiency  of  computers  enable  commanders  and  staff  personnel 
to  spend  their  time  and  mental  energy  on  higher  level  RI  processes  (i.e.,  information  of 
importance  to  the  commander  and  the  unit),  which  leads  to  reduced  uncertainty  and  better 
decisions.  Conversely,  when  misused,  computers  can  produce  large  quantities  of 
irrelevant  data  that  hamper  the  commander’s  ability  to  make  timely  and 
effective  decisions. 

People  are  the  key  components  within  any  C  system.  Even  the  most  advanced 
technology  cannot  support  the  C  system  without  people  (Department  of  the  Anny, 
2003).  Satellites  and  fiber  optics  can  relay  vast  amounts  of  data  across  entire  oceans  and 
continents.  Unmanned  aerial  (UAV)  and  ground  vehicles  (UGV)  can  stream  countless 
hours  of  high-resolution  video.  Remote  sensors,  radar,  and  sonar  can  transmit  multitudes 
of  near  simultaneous  signals.  However,  interfaces  designed  on  an  incorrect 
understanding  of  cognition  will  ultimately  degrade,  rather  than  improve,  performance 
(Klein,  Ross,  Moon,  Klein,  Hoffman,  &  Hollnagel,  2003).  From  an  HSI  perspective,  the 
most  important  concepts  concerning  the  impact  of  advanced  INFOSYS  technology  on 
human  perfonnance  in  the  C  system  lie  in  the  cognitive,  information,  and  physical 
domains  as  described  by  Money  (2001).  For  this  study,  the  goal  is  to  detennine  how 
effective  RAPTOR’S  design  is  at  supporting  a  decision  maker’s  ability  to  separate  fact 
from  fiction  during  fluid  situations. 
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2. 


Command  on  the  Modern  Battlefield 


It  is  important  to  recognize  that  the  speed  and  efficiency  of  INFOSYS 
technologies  both  increase,  as  well  as  degrade,  the  commander’s  ability  to  make 
decisions  on  the  modern  battlefield.  NCO  strives  to  ease  the  burdens  of  command  by 
networking  commanders,  subordinate  leaders,  shooters,  and  battlefield  sensors  together 
through  a  robust,  secure,  and  broadband  “tactical  Internet.”  For  the  first  time  in  the 
history  of  warfare,  warfighters  possess  the  capability  to  share,  analyze,  collaborate,  and 
internalize  distant  battlespace  information  in  near  real-time  (Kemmerer,  2008). 
Commanders  have  the  ability  to  digitally  transmit  (i.e.,  e-mail)  graphical  representations 
of  their  intent,  concepts,  and  directives  across  the  entire  command.  The  end  users  (i.e., 
subordinate  leaders,  individual  Soldiers,  etc.)  also  have  the  ability  to  display,  store,  and 
retrieve  information  as  needed.  The  increased  speed,  efficiency,  flexibility,  and 
reliability  of  modern  communications  enable  commanders  to  decentralize  decision 
making  and  leverage  subordinate  initiative  to  achieve  mission-related  goals. 

In  reality,  many  of  the  INFOSYS  technologies  that  enable  NCO  have  given  little 
consideration  to  the  role  of  the  human  in  the  design  or  implementation  of  the  systems 
(Read,  2007).  It  is  not  surprising  that  the  measures  commonly  used  to  determine 
INFOSYS  effectiveness  center  around  the  processing  power,  storage  capacity,  and 
bandwidth  provided  by  machines.  These  metrics  are  easily  quantified  and  relatively 
simple  to  produce.  On  the  other  hand,  determining  human  cognitive  perfonnance  is 
extremely  challenging  due  to  the  numerous  variables  and  relationships  involved. 
Accurate  measurements  of  human  cognition  require  a  “constellation”  of  individual- 
difference  indices  that  are  time  consuming  to  produce  and  difficult  to  quantify  (Aldag  & 
Power,  1986). 

The  failure  to  recognize  humans  as  the  critical  information  nodes  within 
INFOSYS  designs  ultimately  degrade  total  system  performance  and  further  complicate 
command.  Tactical  operations  are  ongoing  on  the  modem  battlefield.  Commanders 
constantly  receive  influxes  of  new  information  that  alter  what  was  perceived  as  truth  only 
moments  before.  Events  proceed  forward  in  time  at  a  rate  that  can  quickly  outpace  the 
commander’s  ability  to  comprehend  their  meaning  (Phister  et  ah,  2003).  Though 
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technology  can  be  designed  to  provide  countless  terabytes  of  data  at  increasing  speeds 
and  efficiency,  the  warfighter’s  mental  skills,  judgment,  and  expertise  will  always  be 
required  to  determine  the  data’s  relevancy. 

3.  Control  on  the  Modern  Battlefield 

Success  in  command  is  impossible  without  control.  When  exercising  control,  the 
commander  must  understand  the  effects  numerous  interrelated  factors  (e.g.,  enemy 
forces,  terrain,  weather,  time,  etc.)  will  have  on  tactical  operations  as  they  make  decisions 
and  direct  friendly  forces  toward  mission  accomplishment.  NCO  theory  suggests  that 
advanced  INFOSYS  technologies  enable  understanding  by  allowing  commanders  to 
access  any  type  of  data  they  desire  in  near  real-time.  However,  as  stated  by  Shattuck, 
Graham,  Merlo,  and  Hah  (2000),  “technology  often  overwhelms  commanders  by 
providing  them  with  more  information  than  they  can  possibly  use  (p.  116).”  Achieving 
understanding  is  primarily  a  human  activity,  and  cannot  be  attained  with 
technology  alone. 

As  Figure  2  depicts,  achieving  understanding  requires  the  transformation  of 
information  through  four  different  levels  of  meaning  called  the  cognitive  hierarchy 
(Department  of  the  Army,  2003).  Data  is  at  the  lowest  level  of  the  hierarchy  and 
comprises  the  unprocessed  bits  and  bytes  of  information.  Commanders  and  their  staff 
collect,  filter,  and  sort  data  to  determine  relevancy.  Information  is  formed  by  relevant 
bits  of  data  that  are  organized  and  fused  together  to  provide  meaning.  Knowledge  is 
inherently  cognitive,  and  involves  assigning  greater  values  of  meaning  to  information 
through  further  analysis  and  evaluation,  until  potential  implications  on  tactical  operations 
are  recognized  (Garstka  &  Alberts,  2004).  Finally,  understanding  is  achieved  when  the 
commander  comprehends  what  is  happening  and  why,  and  applies  judgment  to  affect  a 
specific  situation’s  inner  relationships  (Department  of  the  Anny,  2003). 
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Figure  2.  The  Cognitive  Hierarchy  (From:  Department  of  the  Army,  2003). 

In  the  context  of  the  cognitive  hierarchy,  information  management  (IM)  is  an 
essential  aspect  of  control.  Staff  personnel  use  IM  processes  to  harvest  and  exhibit  RI  to 
facilitate  the  commander’s  decision-making  process.  Staff  personnel  assist  with  the 
control  function  by  utilizing  RI  outputs  (e.g.,  commander’s  intent,  concepts,  decisions, 
etc.)  to  build  a  common  operating  picture  (COP).  The  COP  serves  as  a  guide  for  all 
echelons  of  command  to  follow  while  working  toward  a  common  goal.  Interfaces  play  a 
crucial  role  in  these  control  activities  by  enabling  staff  personnel  to  display  the  COP  in 
efficient  and  usable  fonnats.  Commanders  and  staff  personnel  then  leverage  INFOSYS 
technologies  and  NCO  networks  to  ensure  the  right  information  is  shared  with  the  right 
people  at  the  right  time  (Garstka  &  Alberts,  2004). 

B.  UNDERSTANDING  THE  INFORMATION  AGE  BATTLEFIELD 

1.  The  Cognitive  Domain  of  Command 

Understanding  enables  the  commander  to  fonn  a  mental  picture  (i.e.,  battlefield 
visualization)  of  the  friendly  force’s  current  state  in  relation  to  the  enemy’s  current  state 
both  in  time  and  space  (Department  of  the  Anny,  2003).  Forming  this  mental  picture 
encompasses  rigorous  cognitive  processes  that  allow  commanders  to: 

•  Make  more  accurate  assessments  for  how  the  environment  will  impact 
operations  (e.g.,  terrain,  weather,  temperature,  light,  time,  etc.). 
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•  Determine  the  current  operational  state  of  friendly  and  enemy  forces  (e.g., 
location,  capabilities,  status  of  resources,  morale,  etc.). 

•  Formulate  alternative  COAs  that  capitalize  on  friendly  force  strengths  and 
enemy  weaknesses. 

•  Direct  forces  in  order  to  achieve  mission  related  objectives  (i.e., 
communicating  intent,  concepts,  and  decisions). 

The  proficiency  at  which  decision  makers  apply  these  cognitive  processes 
depends  on  the  cognitive  domain,  or  mind,  of  the  individual  commander.  This  is  the  part 
of  a  commander’s  brain  where  doctrine,  TTPs,  knowledge,  SA,  intent,  and  decision¬ 
making  skills  reside  (Department  of  Defense,  2001). 

Ultimately,  INFOSYS’  success  in  enhancing  decision-maker  performance  relies 
on  the  integration  of  human  cognition  and  technological  capability  (Read,  2007).  The 
key  to  this  integration  is  a  firm  understanding  of  the  human  cognitive  capabilities  and 
limitations  within  INFOSYS.  A  human’s  thought  process,  as  described  by  Garstka  and 
Alberts  (2004),  is  a  process  of  “sensemaking,”  or  forming  awareness  of  key  elements 
relevant  to  the  mission.  The  process  begins  when  a  person  senses,  or  perceives,  incoming 
stimuli  (i.e.,  data).  The  person’s  cognition  interprets  meaning  by  forming  schemas,  or 
“mental  models,”  to  compare  the  information  against  similar  situations  experienced  in  the 
past.  As  the  person  makes  sense  of  this  mental  picture,  he  evaluates  available  options, 
and  then  decides  what  action  to  take  (Smith,  2006).  Over  time,  people  gain  experiences 
from  which  they  compile  a  repertoire  of  mental  models  that  apply  across  a  range  of 
situations  (Garstka  &  Alberts,  2004). 

However,  as  previously  discussed,  humans  have  limited  cognitive  resources  in 
which  to  comprehend  large  amounts  of  information.  Constant  influxes  of  ambiguous 
and/or  conflicting  information  can  severely  impede  a  person’s  ability  to  make  sense  of 
the  situation.  Commanders  may  succumb  to  cognitive  biases  generated  by  their 
inaccurate  interpretation  of  the  environment  (Garstka  &  Alberts,  2004).  These  biases 
potentially  cause  decision-makers  to  deviate  from  objectivity  and  make  errors  in 
judgment  (Arden,  1996). 
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While  certain  cognitive  activities  (e.g.,  working  memory)  draw  on  limited- 
capacity  resources,  other  activities  (e.g.,  pattern-recognition)  draw  on  virtually  unlimited 
perceptual  resources  (Bennett,  Payne,  &  Walters,  2005).  Garstka  and  Alberts  (2004) 
describe  people’s  cognitive  capabilities  as  very  good  at  identifying  patterns  in  disparate 
information,  making  inferences,  and  learning.  Therefore,  infonnation  complexity  in  itself 
is  not  a  problem,  given  meaningful  information  is  presented  in  a  coherent  and  structured 
manner  (Rasmussen,  1992).  The  essential  notion  being  that  in  order  for  INFOSYS 
technologies  to  improve  total  system  perfonnance  and  effectiveness,  infonnation  must  be 
constructed  into  representations  that  exploit  the  inherent  pattern-recognition  capabilities 
of  the  human,  while  also  decreasing  reliance  on  limited-capacity  resources  (Bennett  et  ah, 
2005).  Fortunately,  as  Rasmussen  and  Vicente  (1990)  argue,  the  power  and  flexibility  of 
information  technology  make  it  possible  for  interfaces  to  adapt  to  human  capabilities 
and  limitations. 

2.  Cognitive  Integration  of  Information 

Shattuck  et  ah,  (2000)  explains  that  “cognitive  integration  occurs  as  commanders 
and  staffs  extract  data  from  disparate  resources  and  combine  them  in  ways  to  create  a 
veridical,  holistic  view  of  the  environment  (p.  117).”  Thus,  the  cognitive  integration 
process  aids  the  commander  in  achieving  understanding.  Commanders  craft  several 
products  such  as  the  commander’s  intent,  commander’s  critical  information  requirements 
(CCIR),  and  planning  guidance  to  describe  their  understanding  (Department  of  the  Army, 

2003) .  Staffs,  in  turn,  use  these  products  to  construct  the  COP. 

The  physical  and  information  domains  provide  the  infrastructural  and 
informational  foundation  for  information  collection  and  integration  (Garstka  &  Alberts, 

2004) .  Money  (2001)  attributes  the  physical  domain  as  the  traditional  domain  of  warfare. 
This  is  where  the  physical  platforms  and  the  communications  networks  that  connect  them 
reside.  The  infonnation  domain  is  where  information  is  created,  codified,  and  shared. 
This  is  also  the  domain  where  the  C  of  modern  military  forces  is  communicated,  and 
where  the  commander’s  intent  is  conveyed  (Department  of  Defense,  2001).  The 
networks  within  the  physical  domain  enable  the  transfer  of  information  packaged  in  the 
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information  domain  (Garstka  &  Alberts,  2004).  As  Figure  3  suggests,  managing  the 
cognitive,  physical,  and  information  domains  while  also  analyzing  and  integrating 
information  extracted  from  nodes  located  throughout  the  battlefield  is  an  extremely 
dynamic  activity. 

Seusemaking  Do,n*i,i 


Figure  3.  C2  Conceptual  Model  (From:  Phister,  Busch,  &  Plonisch,  2003). 

Thus,  cognitive  integration  occurs  in  evolving  contexts  such  as  those  inherent  in 
the  C“  of  tactical  operations  (Shattuck  et  ah,  2000).  For  example,  as  tactical  units  execute 
assigned  tasks  in  the  physical  domain,  they  report  friendly  and  enemy  actions  in  the 
information  domain.  The  commander  and  his  staff  assess  the  causes  for  the  action,  and 
analyze  the  effects  potential  reactions  will  set  in  motion.  Conclusions  drawn  from  the 
analysis  trigger  more  decisions  to  be  made  in  the  cognitive  domain,  which  are 
subsequently  disseminated  in  the  information  domain.  This  action-reaction  sequence 
stimulates  other  agents  to  respond  during  ensuing  cycles,  with  the  cycles  repeated  time 
and  again  in  the  course  of  an  interaction  (Smith,  2006). 

Integrating  data  collected  from  various  technological  sources  and  multiple  agents 
is  an  arena  in  which  C  interfaces  can  provide  considerable  assistance  to  commanders. 
For  example,  shifting  at  least  some  portion  of  the  integration  task  to  a  cooperative 
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machine  agent  frees  the  cognitive  resources  of  decision-makers  to  reason  about  the 
situation  in  a  more  sophisticated  manner.  Second,  interfaces  designed  to  make  salient  the 
data  that  are  most  important  focus  the  commander  on  the  significant  portions  of  the 
battlefield.  Third,  interfaces  can  enable  commanders  to  employ  simplifying  strategies 
such  as  tracking  events  at  higher  levels  of  hierarchy  that  reduce  their  cognitive  load 
(Shattuck  et  ah,  2000).  Also,  in  the  context  of  on-going  operations,  interfaces  can 
provide  feedback  loops  that  enable  commanders  to  determine  whether  planned  actions 
achieved  or  deviated  from  intended  effects  (Smith,  2006).  Finally,  interfaces  can  display 
the  commander’s  battlefield  visualization  in  meaningful  and  structured  representations 
that  subordinate  leaders  and  Soldiers  can  understand  and  use  when  executing  assigned 
tasks  to  achieve  common  goals. 

C.  THE  CONGRUENCE  OF  DECISION  MAKING  AND  SA  IN  C2 

1.  Decision  Making  During  Tactical  Operations 

FM  6-0  (2003)  describes  decision  making  as  selecting  the  one  most  favorable 
COA  to  accomplish  a  mission.  The  United  States  Army’s  traditional  view  toward 
selecting  the  best  COA  is  for  commanders  and  staff  personnel  to  use  structured, 
analytical  processes  to  generate  and  compare  several  alternative  solutions  to  the  problem 
until  a  superior  solution  is  identified  (Department  of  Defense,  2003).  The  Military 
Decision-Making  Process  (MDMP)  is  an  example  of  an  analytical  or  rational  choice 
decision-making  model  routinely  used  at  the  tactical  levels.  MDMP’s  methodical  process 
serves  well  for  decision-making  in  complex  and  unfamiliar  situations  because  it  helps 
commanders  and  staffs  organize  their  thoughts  to  ensure  all  factors  have  been  considered, 
analyzed,  and  evaluated  before  reaching  decisions.  Though  MDMP  assists  commanders 
in  developing  precise  plans  with  minimal  human  error,  its  deliberate  and  time-consuming 
process  is  not  appropriate  during  time-constrained  situations  commonly  encountered 
during  the  execution  phases  of  tactical  operations  (Department  of  Defense,  2003). 

For  the  past  couple  of  decades,  numerous  studies  have  been  conducted  to  gain 
insight  on  how  experienced  practitioners  make  decisions  in  complex  real-world  settings 
characterized  by  time  pressure,  uncertainty,  ill-defined  goals,  and  high  personal  stakes. 
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Decision  making  in  these  types  of  environments  are  commonly  referred  to  as  naturalistic 
decision  making  (NDM),  which  has  been  widely  accepted  in  recent  years  due  to  its  utility 
in  operational  settings  (Shattuck,  2007).  The  latest  versions  of  United  States  Army 
doctrine  recognizes  NDM’s  utility  during  certain  situations,  and  describes  the  process  as 
intuitive  decision  making  (Department  of  Defense,  2003).  Perhaps  most  importantly,  as 
stated  by  Kemmerer  (2008),  “NDM  characteristics  describe  the  situations  and  context  for 
modem  military  exercises  and  engagements”  (p.  9)  (e.g.,  March  2002  combat  operations 
to  destroy  al  Qaeda  and  Taliban  forces  in  Afghanistan’s  Shah-i-Kot  Valley). 

In  examining  the  utility  of  NDM  in  tactical  operations,  Lipshitz,  Klein,  Orasanu, 
&  Salas  (2001)  cite  studies  where  naval  surface  ship  commanders,  tank  platoon  leaders, 
and  infantry  officers  were  used  to  detennine  decision  strategies  normally  employed  by 
proficient  decision  makers  during  complex  and  time  constrained  situations.  Findings 
from  these  studies  suggest  that  proficient  decision-makers  use  their  experience  and 
training  to  pattern  match  appropriate  responses  to  the  given  situation.  They  then  develop 
and  mentally  wargame  one  plausible  COA  rather  than  taking  time  to  deliberately  and 
methodically  contrast  the  COA  with  multiple  alternatives  using  a  common  set  of  abstract 
evaluation  dimensions  (Ross,  Klein,  Thunhohn,  Schmitt,  &  Baxter,  2004).  Therefore, 
NDM  can  be  viewed  as  a  form  of  satisficing,  rather  than  optimization,  since  commanders 
often  decide  on  COAs  that  are  simply  “good  enough”.  Although  a  number  of  models  fall 
within  the  NDM  framework,  Klein’s  Recognition  Primed  Decision-Making  (RPD)  model 
serves  as  the  prototypical  NDM  model  (Lipshitz  et  al.,  2001). 

The  RPD  model  was  developed  on  the  basis  that  while  under  time  pressure, 
commanders  rely  on  past  experiences  to  select  their  COA  rather  than  generating  a  large 
set  of  options  (Klein,  1993).  The  RPD  model  (see  Figure  4)  currently  consists  of  three 
variations  depending  on  the  familiarity  of  the  situation.  During  familiar  situations, 
skilled  decision  makers  can  usually  generate  a  feasible  COA  as  the  first  one  they 
consider.  During  settings  where  the  situation  is  not  clear,  the  decision  maker  will  often 
rely  on  a  story-building  strategy  to  mentally  simulate  (i.e.,  wargame)  events  and  construct 
one  solution  more  plausible  than  another.  In  the  third  variation,  the  decision  maker  can 
employ  a  “progressive  deepening”  strategy  to  anticipate  whether  the  COA  will  succeed  or 


18 


fail  by  determining  if  unacceptable  consequences  exist.  If  unacceptable  consequences  do 
potentially  exist,  the  decision  maker  can  continue  to  mentally  wargame  events  until  a 
reasonable  alternative  is  identified  (Lipshitz  et  al.,  2001). 
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Figure  4.  Recognition  Primed  Decision  Model  (From:  Klein,  2008). 

FM  6-0  (2003)  stresses  that  analytical  and  intuitive  decision-making  are 
complementary  thought  processes.  The  use  of  one  process  over  another  depends  on  time 
available  and  the  nature  of  the  specific  situation.  Hamm  (1988)  even  proposes  that  a 
proficient  decision-maker’s  thinking  will  often  shift  between  analysis  and  intuition,  while 
never  releasing  his  hold  on  either  of  the  two.  Nevertheless,  most  analytical  models  tend 
to  ignore  experience  and  perception  as  critical  variables  in  decision  making,  while  NDM 
models  place  them  at  the  center  of  interest.  The  emphasis  on  perceptual  processes  and 
dynamic  action  constraints  in  decision  making  has  increased  the  awareness  of  the 
potential  role  C2  interfaces  can  play  in  providing  effective  decision  support  to 
commanders  (Bennett  et  al.,  2008).  However,  while  the  aforementioned  models  describe 
general  processes  of  decision  making  during  tactical  operations,  they  do  not  address  the 
role  perception  plays  at  shaping  a  commander’s  perspective  for  how  a  given  decision 
may  potentially  influence  specific  events. 
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2.  The  Function  of  Perception  in  Dynamic  Environments 

Perception  is  an  active  process  of  inference  in  which  a  person  constructs  reality 
from  raw  data  collected  by  the  senses  (Arden,  1996).  In  the  context  of  dynamic 
environments,  the  most  important  aspect  of  perception  is  the  extent  to  which  the  process 
enables  a  commander  to  accurately  forecast  how  current  events  will  potentially  impact 
future  goals,  objectives,  and  end  states.  The  perception  of  elements  within  a  volume  of 
time  and  space,  the  comprehension  of  their  meaning,  and  the  projection  of  their  status  in 
the  near  future  is  commonly  referred  to  as  situational  awareness  (Endsley,  1995).  In 
tactical  situations,  accurate  SA  facilitates  flexible  and  agile  forces  that  are  capable  of 
acting  faster  than  the  enemy  (Bushey  &  Forsyth,  2006). 

SA  is  a  dynamic  and  multifaceted  concept  that  has  been  associated  with  complex 
military  systems  since  the  tenn  was  first  introduced  by  United  States  Air  Force  pilots 
during  the  Vietnam  War.  FM  1-02  defines  SA  as  knowledge  and  understanding  of  the 
current  situation  which  promotes  timely,  relevant,  and  accurate  assessments  of  friendly, 
enemy,  and  other  operations  within  the  battlespace.  The  situations  of  most  concern  to  the 
warfighter’s  SA  are  those  which  can  vary  rapidly,  and  which  the  commander  is 
responsible  for  managing  through  decisive  action.  To  be  of  value,  the  awareness  of  a 
situation  of  concern  must  cover  all  relevant  factors,  be  up-to-date,  be  expertly  interpreted, 
and  capture  the  real  meaning  and  full  implications  of  the  situation  (McGuinness,  2004). 
Endsley  (1995)  portrays  a  warfighter’s  situation  valuation  process  through  three  distinct 
levels  of  SA: 

•  Level  1  -  perception  of  elements  in  the  environment 

•  Level  2  -  comprehension  of  the  current  situation 

•  Level  3  -  prediction  of  the  future  actions  of  data  elements 

Success  at  higher  levels  of  SA  depends  on  a  person’s  knowledge  of  events  during 
the  lower  levels  of  SA.  For  example,  a  commander  may  perceive  a  deviation  in  the 
planned  action  of  a  subordinate  element,  comprehend  how  the  deviation  may  endanger 
task  achievement  to  potential  enemy  counter-actions,  understand  when  and  where  the 
future  contact  will  take  place,  and  finally,  predict  how  serious  the  outcome  may  be.  The 
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diagnostic  distinction  between  the  three  levels  is  important  because  breakdowns  in  a 
perceptual/cognitive  operation  will  possess  very  different  consequences  for  addressing 
each  level  (Wickens,  2008). 

It  is  important  to  point  out  that  accurate  decision  making  does  not  rely  solely  on  a 
commander’s  achieved  level  of  SA.  It  is  entirely  possible  for  a  commander  to  have 
excellent  SA  and  still  disseminate  poor  directives  because  he  may  lack  the  requisite 
knowledge  to  implement  corrective  procedures  aimed  at  remedying  the  situation. 
Likewise,  it  is  also  possible  for  a  commander  with  minimal  SA  to  implement  timely  and 
accurate  decisions  because  his  experience  and  training  may  be  sufficient  enough  to  offset 
his  degraded  view  of  the  situation  (Adams,  Tenney,  &  Pew,  1995).  Hence,  it  is  worth 
noting  that  a  commander  must  still  understand  the  task  demands  regardless  of  the  level  of 
SA  he  has  achieved.  Therefore,  in  the  context  of  dynamic  environments,  perhaps  the 
most  important  concept  of  SA  may  be  the  degree  to  which  the  process  facilitates  a 
commander’s  coordination  of  his  own  perception,  decision  making,  and  action  loop 
(Flach,  1995). 

Like  any  complex  work  domain  that  relies  on  technology  to  assist  operators  as 
they  cope  with  novel  situations,  the  United  States  Military  strives  to  develop  interfaces 
that  enhance  SA  and  increase  operator  perfonnance.  Though  seemingly  simplistic,  the 
tools  often  provided  to  enhance  SA  are  no  longer  simple;  they  are  amazingly  intricate  and 
require  operators  to  perform  elaborate  perceptual  and  cognitive  tasks  (Endsley  & 
Garland,  2000).  As  such,  acquiring  and  maintaining  high  levels  of  SA  must  be 
appreciated  as  an  integral  part  of  the  operator’s  mental  workload  (Adams  et  ah,  1995). 
An  increase  in  workload  can  divert  scarce  cognitive  resources  from  maintaining  SA, 
while  a  well-designed  usable  display  can  both  reduce  workload  and  increase  SA 
(Wickens,  2008).  Thus,  when  evaluating  the  scale  to  which  new  technological  design 
concepts  actually  improve  (or  degrade)  operator  SA,  it  is  imperative  to  systematically 
evaluate  them  based  on  a  measure  of  SA  (Endsley,  1995). 

The  magnitude  to  which  evolving  technology  affects  the  man-machine  symbiosis 
must  be  highlighted  as  a  primary  concern  due  to  increasing  dependence  humans  place  on 
the  use  of  computers  and  other  automated  tools.  When  determining  an  interface’s 
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effectiveness  at  assisting  the  warfighter  with  gaining  understanding,  maintaining  SA,  and 
decision  making  processes,  one  must  apply  a  model  that  includes  both  man  and  machine. 
The  model  that  best  portrays  this  necessary  relationship  is  the  Dynamic  Model  of 
Situated  Cognition  (DMSC). 

3.  Situated  Cognition  in  Fluid  Real-World  Settings 

The  tenets  of  NCO  presume  that  a  robustly  networked  force  will  ultimately  result 
in  dramatically  improved  mission  effectiveness  facilitated  by  enhanced  decision-making 
and  increased  levels  of  SA  (Garstka  &  Alberts,  2004).  A  2002  study  conducted  by 
Maritime  Systems  Action  Group  One  (MAR  AG-1)  specifically  examined  the 
“exponential”  increases  in  mission  effectiveness  claimed  for  NCO  technologies.  Results 
of  the  study  led  MAR  AG-1  to  conjecture  that  a  change  in  the  use  of  technology  (rather 
than  a  change  in  technology  itself)  is  required  since  the  positive  effects  of  technology  on 
human  behavior  is  difficult  to  validate  in  all  but  the  most  simplistic  of  circumstances 
(Hazen,  Burton,  Klingbeil,  Sullivan,  Fewell,  Grivell,  Philp,  &  Marland,  2003).  Thus,  the 
challenges  associated  with  integrating  technological  systems  adept  at  supporting  a 
warfighter’s  decision-making  and  SA  in  fast-paced  and  dynamic  environments  continues 
to  remain  a  central  concern  within  the  United  States  military. 

DMSC  represents  this  integration  of  man  and  machine  in  tactical  operations,  and 
illustrates  how  human  decision-making  processes  are  influenced  by  technological  agents. 
DMSC  achieves  this  by  providing  a  model  that  couples  NDM  theory  (see  Chapter  II.C.l.) 
with  a  conceptual  model  that,  unlike  RPD,  includes  technology  to  provide  a  more  robust 
insight  into  total  system  performance  (Shattuck  &  Miller,  2006).  Simply  stated,  in 
addition  to  the  characteristics  of  individuals,  DMSC  takes  into  account  that  the  design  of 
an  interface  can  also  affect  SA  and  decision  making  by  representing  the  environment 
more  or  less  accurately  (Endsley,  1995;  Shattuck  &  Miller,  2006).  Thus,  before 
discussing  how  interface  designs  can  help  commanders  cope  with  the  speed,  uncertainty, 
and  ill-structured  situations  inherent  on  the  modem  battlefield,  one  must  first  understand 
the  inextricable  links  that  exist  between  the  technological  components  and  the  human 
agents  in  the  C2  system  (Shattuck  &  Miller,  2006). 
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DMSC  (see  Figure  5)  employs  a  process  tracing  technique  (i.e.,  uses  multiple  data 
collection  methods  throughout  the  man-machine  system)  to  assess  human-system 
performance  (FISP).  This  assessment  is  conducted  by  mapping  a  decision-maker’s 
cognition  (i.e.,  perception,  interpretation,  understanding,  etc.)  as  events  within 
operational  settings  unfold  (Shattuck  &  Miller,  2006). 
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Figure  5.  Dynamic  Model  of  Situated  Cognition  (From:  Miller  &  Shattuck,  2006). 


a.  Technological  Aspects  of  DMSC 

Oval  1  depicts  everything  in  the  environment.  The  various  shapes  and 
colors  represent  individual  data  elements  located  throughout  the  battlespace  (i.e.,  terrain, 
weather,  enemy,  friendly,  civilians,  etc.).  Oval  1  can  be  referred  to  as  ground  truth,  or  as 
a  “God’s  eye  view”  of  reality  (Shattuck  &  Miller,  2006). 

Oval  2  depicts  those  data  elements  accurately  detected  by  battlefield 
sensors.  Oval  2  contains  only  a  subset  of  data  from  Oval  1  since  it  is  impossible  for  even 
the  most  sophisticated  array  of  technological  systems  to  detect  everything  that  exists 
within  the  environment. 

Oval  3  represents  data  displayed  on  an  operator’s  screen.  Oval  3  is  an 
even  narrower  subset  of  data  due  to  the  inaccuracies  propagated  by  faulty  sensors  in 
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Oval  2.  Misrepresentations  can  also  occur  when  data  is  fused  by  flawed  technological 
nodes  linking  Ovals  3  and  2  together,  or  can  be  misrepresented  by  poorly  designed 
displays  (Shattuck  &  Miller,  2006). 


b.  Human  Aspects  of  DMSC 

The  model  incorporates  three  lenses  (labeled  A,  B,  and  C  in  Figure  4)  that 
mediate  how  information  is  processed  by  the  decision-maker  (Read,  2007).  As  can  be 
seen  in  Figure  6,  the  lenses  focus  attention  toward  certain  data,  and  in  some  cases  change, 
skew,  and  even  bias  how  a  commander  perceives,  comprehends,  and  makes  projections 
as  information  passes  through  his  individual  lenses. 


•  Mler  ana  Shorajck. 


Figure  6.  Integration  of  Distorted  Information  (From:  Miller  &  Shattuck,  2006). 


According  to  the  model,  Lens  A  directs  the  commander’s  attention  to 


selected  incoming  stimuli  (e.g.,  visual  and  auditory)  from  Oval  3.  Oval  4  represents  an 
even  smaller  subset  of  data  perceived  by  the  commander.  Shattuck  and  Miller  (2006) 
describe  this  perception  process  in  terms  of  passive  or  active  input.  Active  input  can  be 
considered  as  specific  information  requested  by  the  commander,  while  passive  input  is 
non-requested  information.  Numerous  factors  (e.g.,  social  culture,  operational  goals, 
guidelines,  training,  experience,  and  fatigue)  contribute  to  the  narrowing  perception  of 
data  by  influencing  which  stimuli  a  commander  focuses  his  attention  on. 
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Oval  5  represents  comprehension  (i.e.,  understanding)  of  the  information, 
while  Oval  6  represents  the  commander’s  projections  (i.e.,  prediction).  Lenses  B  and  C 
are  impacted  by  the  same  factors  that  directed  the  commander’s  attention  and  perception 
(Shattuck  &  Miller,  2006).  By  Oval  5,  the  commander  will  have  made  decisions  and 
issued  directives  based  on  the  way  his  cognitive  processes  fused,  processed,  and 
organized  infonnation  filtered  by  Lens  B.  Oval  6  (Projection)  is  depicted  by  a  larger 
broken  border  that  illustrates  the  commander’s  mental  model  for  what  he  believes  to  be 
true,  and  his  projections  for  how  future  events  will  unfold.  It  is  important  to  note  that  the 
amorphous  shapes  surrounding  Ovals  5  and  6  represent  varying  interpretations  of 
information  (Shattuck  &  Miller,  2006).  Though  numerous  commanders  may  receive  the 
same  data,  their  interpretations,  decisions,  and  predictions  may  differ  along  varying 
degrees  as  their  individual  lenses  influence  their  understanding  and  levels  of  SA. 

c.  Technological  Influences  on  Decision-Making  and  SA 

DMSC  also  incorporates  feedback  loops  (see  Figure  7)  to  provide  insight 
into  the  cognitive  processing  and  decision-making  of  a  practitioner  (Shattuck  &  Miller, 
2006).  SA  and  decision-making  is  an  iterative  process  that  evolves  throughout  a 
perception-action-reaction  cycle,  and  is  represented  in  Ovals  2  to  5.  Understanding 
enables  a  commander  to  make  projections  for  how  he  expects  events  to  unfold.  The 
commander  may  reorient  battlefield  sensors  (e.g.,  UAV,  UGV,  etc.)  to  confirm  or  deny 
his  expectations.  This  decision  is  represented  by  the  feedback  loop  from  Oval  6  to  Oval 
2.  Additional  data  collected  by  sensors  flow  from  Oval  2  to  Ovals  3,  4,  5,  and  6 
(Shattuck  &  Miller,  2006). 
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Figure  7.  Feedback  Loops  in  the  DMSC  (From:  Miller  &  Shattuck,  2006). 

These  feedback  loops  highlight  important  considerations  for  C"  interface 
designs.  First,  data  must  be  presented  (Oval  3)  in  a  coherent  and  structured  manner  so 
that  the  operator  can  focus  sufficient  cognitive  resources  (Oval  4)  to  accurately  interpret 
(Oval  5)  meaning  (Rasmussen,  1992).  Next,  perceptual  cues  provided  by  the  interface 
display  must  be  salient  enough  to  assist  an  operator  at  determining  how  they  expect 
events  to  unfold  (Oval  6)  (Bennett  et  ah,  2005).  Finally,  information  represented  on  the 
interface  display  must  be  sufficiently  robust  so  that  an  operator  can  identify  and  mitigate 
uncertainty  by  refining  sensor  inputs,  updating  technological  outputs,  or  changing  their 
cognitive  approach  (Ovals  2  through  4)  (Kemmerer,  2008). 

These  considerations,  as  well  as  the  numerous  other  human  system 
integration  challenges  described  during  the  preceding  discussions,  set  the  conditions 
required  to  adequately  portray  how  RAPTOR’S  design  improves  total  system 
performance  by  supporting  human  capabilities  and  limitations. 
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D. 


THEORETICAL  FOUNDATIONS  OF  RAPTOR 


1.  Cognitive  Systems  Engineering  (CSE)  Design  Framework 

The  RAPTOR  interface  employs  a  CSE  approach  to  assist  military  practitioners  in 
executing  C  during  tactical  operations.  CSE  provides  the  overarching  framework, 
concepts,  and  analytical  tools  that  can  be  used  to  guide  the  development  of  intuitive  and 
highly  graphical  interfaces  (Bennett  et  ah,  2008).  Unlike  design  schemes  used  for  many 
expert  tools,  CSE  emphasizes  interface  designs  that  support  knowledgeable  professionals 
by  keeping,  rather  than  replacing,  the  human  in  the  loop  (Potter,  Elm,  Roth,  &  Woods, 
2001).  Consequently,  RAPTOR’S  design  methodology  seeks  to  capitalize  on  a  military 
practitioner’s  experience,  training,  and  knowledge  by  utilizing  technology  to  transform 
decision-making  from  a  cognitive  activity  to  a  perceptual  activity  (Bennett  & 
Zimmerman,  2001).  Therefore,  the  CSE  process  must  be  reviewed  to  appreciate  how 
RAPTOR’S  design  will  enable  military  commanders  to  effectively  see  themselves,  the 
terrain,  and  the  enemy. 

An  effective  CSE  process  must  consider  the  three  mutually  interacting 
“behavioral-shaping”  constraints  of  domain,  agent,  and  interface  in  the  design  of  a  system 
(Bennett  et  ah,  2008).  These  constraints  have  been  discussed  at  great  length  throughout 
the  preceding  sections  of  this  study,  but  can  be  summarized  accordingly: 

•  Domain  -  goals,  laws,  physical,  and  functional  considerations  that  he 
within  complex  work  domains. 

•  Agent  -  cognitive/perception/action  capabilities  and  limitations  of  human 
agents  conducting  the  specific  work  domain  task  requirements. 

•  Interface  -  functionality/design  characteristics  that  introduce  various 
resource  demands  on  users. 

As  Figure  8  illustrates,  the  connections  between  these  component  constraints  must 

be  properly  mapped  in  order  for  an  interface  to  provide  users  with  robust  cognitive 

support  as  they  execute  multifarious  tasks  inherent  in  complex  work  domains  (Potter 

et  ah,  2001).  Thus,  a  fundamental  premise  of  CSE  is  that  a  detailed  analysis  of  the  work 

to  be  accomplished  within  a  domain  of  application  is  critical.  Therefore,  CSE  provides 

analytical  tools  (the  abstraction  and  aggregation  hierarchies)  to  identify  and  thread 
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fundamental  component  connections  together  during  work  domain  analyses  (Bennett  & 
Zimmerman,  2001).  Results  of  these  analyses  drive  designs  for  the  informational  content 
that  must  be  presented  by  interface  displays. 


Figure  8.  Structure  of  Interface  Design  Problem  (From:  Vicente  &  Rasmussen, 

1992). 


a.  Abstraction  Hierarchy 

The  abstraction  hierarchy  is  a  useful  analytical  tool  for  representing  a 
work  domain  in  a  way  that  is  relevant  to  interface  designs  (Vicente  &  Rasmussen,  1992). 
As  previously  discussed,  in  the  C2  system  there  are  end-states  (e.g.,  goals,  objectives, 
etc.)  that  must  be  achieved,  and  finite  resources  (e.g.,  functional  and  physical  means)  that 
can  be  used  to  achieve  those  objectives.  The  abstraction  hierarchy  describes  these 
“means-end”  relationships  that  exist  between  goal  and  resource  constraints  along  each 
level  of  the  hierarchy.  The  general  characteristics  comprising  each  abstraction  level  in  a 
C2  system  are: 

•  Functional  Purpose  -  to  synchronize  combat  resources  and  BOS  elements 
to  achieve  mission  accomplishment. 
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•  Abstract  Function  -  involves  the  appropriate  allocation  and  expenditure 
of  finite  combat  resources  (e.g.,  people,  equipment,  technology,  and 
logistics)  to  achieve  objectives  and  goals. 

•  Generalized  Function  -  comprises  the  numerous  functions  and  activities 
performed  by  commanders  and  staff  personnel  during  the  C  of  tactical 
operations.  These  tasks  include  gaining  understanding,  constructing  the 
COP,  making  decisions,  issuing  directives,  projecting  future  events, 
managing  information,  etc. 

•  Physical  Function  -  requires  an  understanding  of  the  capabilities  and 
limitations  of  physical  elements  located  in  the  environment  (friendly  and 
enemy).  Examples  include  effective  ranges  of  weapon  systems,  cruising 
speed  of  vehicles,  sensitivity  of  sensors,  payload  capacity  of  aircraft, 
killing  radius  of  munitions,  physical  fitness  of  Soldiers,  bandwidth  of 
INFOSYS,  etc. 

•  Physical  Form  -  requires  an  understanding  for  where  physical  elements 
are  located  throughout  the  battlespace  (friendly  and  enemy).  Also  requires 
an  understanding  for  the  effects  numerous  interrelated  factors 
(e.g.,  distances,  terrain,  weather,  temperature,  time,  light,  etc.)  will  have 
on  these  elements  during  tactical  operations. 

The  resulting  descriptions  allow  constraints  to  be  mapped  in  terms  of 
reasons,  causes,  and  effects  that  are  nested  upwards  and  downwards  through  the 
abstraction  hierarchy  (Bennett  et  ah,  2008).  An  important  property  of  this  mapping 
technique  is  that  higher  levels  of  the  hierarchy  require  less  detailed  representations  than 
lower  levels  of  the  hierarchy.  As  faults  occur  in  the  lower  levels  of  the  hierarchy,  their 
causes  and  effects  propagate  upward  through  the  hierarchy,  while  the  reasons  for  the  fault 
propagate  back  downwards  to  the  lower  levels  of  the  hierarchy.  The  outputs  of  this 
mapping  technique  result  in  two  important  benefits:  it  provides  operators  with  an 
informational  basis  for  coping  with  unanticipated  events,  and  provides  a  psychologically 
valid  representation  for  problem  solving  (Vicente  &  Rasmussen,  1992). 
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Consequently,  the  abstraction  hierarchy  provides  a  design  technique  that 
allows  RAPTOR  to  capture  critical  data  pertaining  to  the  goals,  purposes  and  constraints 
of  tactical  operations,  and  then  represents  their  information  in  the  form  of  icons,  graphs, 
charts,  and  tables  directly  on  the  display  (Talcott,  Bennett,  Martinez,  Shattuck,  & 
Stansifer,  2007).  As  Figure  9  illustrates,  these  representations  are  arrayed  either  on,  or 
along  a  contour  map  of  the  battlespace,  which  aids  the  operator  in  interpreting  their 
meaning  in  time  and  space.  Infonnation  presented  in  this  manner  provides  visual 
salience,  which  enables  the  commander  to  focus  attention  on  the  priority  measures 
that  have  a  significant  impact  on  operations  (e.g.,  combat  power,  resource  status,  weapon 
range  envelopes,  force  ratios,  distances,  terrain,  time,  etc.).  Also,  certain  symbols  are 
designed  to  change  colors  as  events  evolve  (e.g.,  combat  resource  icons,  combat  resource 
displays,  control  tree,  etc.).  These  changing  colors  represent  either  planned  expenditures 
over  time,  or  faults  propagating  upward  through  the  hierarchy.  Therefore,  not  only  does 
RAPTOR’S  design  help  to  make  salient  the  data  that  are  most  important,  but  also  provide 
the  commander  with  feedback  loops  that  assist  him  in  determining  whether  actions 
achieved  or  deviated  from  intended  effects.  In  short,  RAPTOR’S  representations  of 
critical  cues  in  time  and  space  can  help  a  commander  to  cope  with  complex,  dynamic, 
and  novel  situations  by  simplifying  his  overall  sense  of  real-world  problems 
(Smith,  1989). 
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Figure  9.  Overview  of  RAPTOR  Interface. 


b.  Aggregation  Hierarchy 

The  aggregation  hierarchy  is  an  analytical  tool  used  to  provide  models  for 
the  “part-whole”  structure  of  a  domain  (Bennett  et  ah,  2008).  As  stated  previously,  the 
commander  is  overall  responsible  for  C  and  must  understand  the  situation  before  making 
decisions.  However,  C2’s  complexity  makes  it  impossible  for  a  commander  to  view,  or 
even  consider,  the  system’s  entire  range  of  subcomponents  and  functions  simultaneously. 
Fortunately,  the  aggregation  hierarchy  provides  a  mechanism  for  coping  with  this 
complexity  by,  as  Vicente  and  Rasmussen  (1992)  succinctly  describe,  allowing  the 
commander  to  “see  the  forest  through  the  trees  (p.  593).” 

An  important  premise  of  the  aggregation  hierarchy  is  that  higher  level 
(i.e.,  less  detailed)  forms  of  representation  are  easier  to  comprehend  and  more  efficient  to 
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manage  than  lower  level  (i.e.,  more  detailed)  representations  (Vicente  &  Rasmussen, 
1992).  A  simple  analogy  for  this  premise  can  be  found  in  automobile  warning  lights.  For 
example,  a  driver  notices  that  a  fault  has  occurred  when  the  “check  gauges”  light 
illuminates  on  his  dashboard.  A  quick  examination  of  the  gauges  reveals  the  faulty 
function  stemming  from  high  engine  temperatures.  The  driver  stops  his  vehicle  and 
continues  his  investigation  by  inspecting  only  those  components  that  are  functionally 
responsible  for  cooling  the  engine  during  operation.  This  is  an  efficient  search  since  all 
components  and  subcomponents  not  related  to  the  engine  cooling  system  can  be  ignored. 

Accordingly,  RAPTOR’S  display  presents  icons,  graphs,  charts,  and  tables 
at  higher  and  intermediate  levels  of  visual  salience.  Infonnation  presented  in  higher 
levels  of  hierarchy  reduces  a  commander’s  cognitive  load  by  allowing  him  to  employ 
simplifying  strategies  while  monitoring  battlefield  events.  The  intennediate  level 
representations  cue  the  commander’s  attention  toward  faults  that  potentially  jeopardize 
goals  and  objectives.  Lower  levels  of  detailed  data  are  also  updated,  but  these  data  are 
only  available  when  “drilled-down,”  or  accessed,  by  the  operator.  This  design 
characteristic  automates  a  portion  of  the  infonnation  integration  tasks,  which  frees 
additional  cognitive  resources  that  enable  the  commander  to  reason  about  situations  in  a 
more  sophisticated  manner  (Talcott  et  al.,  2007).  Furthennore,  RAPTOR’S  display 
design  arranges  the  various  forms  of  information  in  a  vertical  and  horizontal  array  on  a 
single  screen.  This  arrangement  supports  the  perception  of  information  in  time  and 
space,  and  reduces  the  operator’s  workload  by  decreasing  the  need  to  divide  attention 
between  multiple  sets  of  tools.  Hence,  decomposed  representations  of  the  C"  system, 
displayed  in  a  coherent  and  structured  manner,  ultimately  enables  commanders  to 
problem  solve  in  the  same  economic  and  proficient  fashion  as  our  fictional  driver. 

c.  Skills,  Rules,  Knowledge  (SRK)  Taxonomy 

While  the  abstraction  and  aggregation  hierarchies  provide  models  for 
mapping  work  domain  constraints  to  interface  designs,  the  SRK  taxonomy  provides  a 
guide  for  communicating  the  domain  design  structure  and  information  content  to  users. 
As  previously  discussed  in  Chapter  II.B.l.,  operators  must  cope  with  the  demands  of  the 
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domain  by  utilizing  their  limited  cognitive  resources.  Therefore,  it  is  imperative  that  an 
interface  design  strategy  takes  advantage  of  the  most  powerful  resources  that  people  have 
for  dealing  with  complexity.  The  SRK  taxonomy  provides  a  useful  framework  for 
capturing  the  various  mechanisms  people  possess  for  information  processing  (Vicente  & 
Rasmussen,  1992). 

Skilled  decision  makers  tend  to  use  their  experience  and  training  to 
determine  the  relevancy  for  events  transpiring  during  fluid  and  time-constrained 
situations  (Lipshitz  et  ah,  2001).  Though  Hamm  (1988)  agrees  that  perception  is  closely 
related  to  good  performance,  findings  from  his  studies  suggest  that  the  skilled  decision 
maker  often  shifts  between  intuitive  and  analytical  modes  of  thinking  when  engaged  in 
problem-solving  activities  (see  Chapter  II.C.l.).  Therefore,  SRK’s  most  important 
concept  is  that  it  emphasizes  the  incorporation  of  both  modes  of  cognitive  control  into 
interface  designs  since  perception  is  not  always  superior  to  analysis. 

In  simple  terms,  RAPTOR  displays  information  in  accordance  with 
standard  United  States  Army  symbols  and  icons  that  represent  “signals”  existing  in  the 
environment  (see  Figure  9).  These  signals  provide  “affordances,”  or  actions,  the  operator 
must  execute  (Bennett  et  ah,  2008).  As  illustrated  in  Figure  9,  signal  representations  are 
arrayed  on  a  contour  map  of  the  battlespace,  which  further  aids  the  operator  in 
interpreting  their  meaning  in  time  and  space.  This  design  feature  takes  advantage  of  the 
commander’s  skilled-based  behavior  by  enabling  him  to  monitor  the  status  of  battlefield 
events  while  exerting  limited  conscious  control. 

Other  informational  representations  presented  in  the  temporal  and  spatial 
synchronization  displays  provide  a  set  of  explicit  rule-based  actions  to  be  followed  during 
the  execution  of  tactical  operations  (see  Figure  9).  Successful  rule-based  behavior 
requires  the  operator  to  recognize  previously  devised  cues  (e.g.,  maneuver  toward  a 
specific  terrain  feature),  and  a  conscious  choice  regarding  the  appropriate  behavior  when 
executing  actions  at  those  cues  (e.g.,  transition  to  a  different  maneuver  formation) 
(Bennett  et  ah,  2008).  In  short,  commanders  can  use  RAPTOR’S  perceptual  signals, 
cues,  and  their  experience  to  quickly  determine  how  they  expect  events  to  unfold 
(Bennett  et  ah,  2008). 
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Knowledge-based  behaviors  involve  situations  that  have  not  been 
previously  encountered  or  accounted  for  during  prior  planning  sessions.  These  novel 
situations  require  the  commander  to  use  limited  capacity  resources  as  he  devises  alternate 
COAs  and  considers  probabilities  of  success  for  each  response.  RAPTOR’S  symbolic 
representations  of  graphical  infonnation  (e.g.,  topographical  map,  combat  resource 
display,  primary  munitions  envelopes,  control  tree  display,  etc.)  eases  this  burden  by 
providing  normative  externalized  mental  models  of  processes  that  can  support  the 
commander  as  he  pattern  matches  and  mentally  simulates  responses.  Thus,  RAPTOR’S 
use  of  robust  representations  exploit  the  military  practitioner’s  experience  and  training, 
while  also  aiding  in  his  analysis  and  decision  making  (Potter  et  ah,  2001). 

2.  Ecological  Interface  Design  Principles 

The  CSE  portions  of  this  chapter  discussed  how  RAPTOR  accounts  for  and 
leverages  behavioral-shaping  constraints  inherent  in  the  work  domain  and  the  agent.  EID 
is  a  branch  of  CSE  that  specifically  addresses  the  behavioral-shaping  constraints  inherent 
in  interface  designs  (Bennett  et  ah,  2008). 

It  is  critical  to  understand  that  interfaces  designed  on  an  incorrect  understanding 
of  cognition  will  ultimately  degrade,  rather  than  improve,  performance  (Klein  et  ah, 
2003).  Thus,  any  interface  designed  to  support  humans  as  they  execute  tasks  in  complex 
work  domains  must  reduce  the  amount  of  cognitive  demands  placed  on  the  operator.  EID 
aims  to  accomplish  this  by  making  interfaces  transparent  and  highly  intuitive  to  operate 
(Rasmussen  &  Vicente,  1990).  The  following  sections  illustrate  how  the  EID  principles 
of  direct  perception,  direct  manipulation,  and  the  perception-action  loop  have  been 
incorporated  into  RAPTOR’S  design  to  transfonn  the  interaction  requirements  associated 
with  decision  making  and  problem  solving  from  cognitive  activities  to  perceptual-motor 
activities  (Bennett  et  ah,  2008). 

a.  Direct  Perception 

Rasmussen  and  Vicente  (1990)  assert  that  “interface  designs  must  take 
advantage  of  the  human’s  remarkable  perception  and  action  capabilities  (p.  101).”  The 
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direct  perception  of  signals  and  cues  decrease  cognitive  resources  and  mental  effort 
required,  which  allows  the  operator  to  use  visual  perceptual  skills  to  make  sense  of  events 
occurring  in  the  battlespace.  Enabling  direct  perception  of  events  represented  by 
interface  designs  requires  at  least  two  sets  of  mappings  (Talcott  et  ah,  2007).  Gibson,  as 
quoted  by  Rasmussen  (1992),  said  the  first  set  (i.e.,  content  mapping)  requires  “the 
designer  to  create  a  virtual  ecology”  by  mapping  the  means-end  relationships  that  exist 
between  domain  constraints  “in  such  a  way  that  the  user  can  read  the  relevant  affordances 
for  actions  (p.  99).”  In  other  words,  content  mapping  encodes  information  content  from 
all  levels  of  the  abstraction  hierarchy  in  the  form  of  graphical  representations. 
Essentially,  these  representations  mimic  signals  and  cues  an  observer  would  encounter 
while  operating  in  the  natural  environment. 

However,  designers  must  also  guard  against  creating  graphical 
representations  in  such  complexity  that  they  confound  the  observer’s  ability  to 
comprehend  their  meaning.  Therefore,  the  second  set  of  mapping  (i.e.,  form  mapping) 
involves  the  relationship  between  the  visual  properties  of  the  graphical  representations 
and  the  perceptual  capabilities  and  limitations  of  the  observer  (Talcott  et  ah,  2007).  Form 
mapping  results  in  representations  that  present  the  part-whole  structure  of  a  domain, 
which  allows  the  operator  to  employ  simplifying  strategies  (e.g.,  chunking)  as  they  make 
sense  of  the  overall  problem  space.  Therefore,  interface  designs  must  facilitate  the 
observer’s  ability  to  determine  the  appropriate  control  actions  to  be  executed  based  on  the 
types  of  signals  and  cues  perceived.  Designs  that  allow  observers  to  choose  between 
varying  levels  from  which  to  view  graphical  representations  make  it  easier  to  perceive 
these  signals  and  cues.  The  following  discussion  explicitly  illustrates  how  RAPTOR’S 
rich  set  of  graphical  representations  incorporate  content  and  form  mapping  design 
principles  that  leverage  the  human’s  perception  and  action  capabilities. 

(1)  Spatial  Synchronization  Display.  As  can  be  seen  in 
Figure  9,  the  left  side  of  RAPTOR’S  display  represents  coarser  (i.e.,  high)  levels  of  detail, 
while  the  right  side  represents  finer  (i.e.,  intennediate)  levels  of  detail  (Bennett  et  ah, 
2008).  Figure  10  illustrates  higher  level  representations  of  physical  elements  (terrain, 
friendly  forces,  enemy  forces,  and  synchronization  points)  arrayed  on  a  topographical 
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map.  These  provide  the  commander  with  salient  visual  representations  of  signals  existing 
in  the  environment,  and  cues  for  actions  that  must  be  executed.  Visual  changes  in  the 
display  (e.g.,  icon  color  changes)  indicate  either  predetermined  combat  resource 
expenditures,  or  faults  propagating  upward  through  the  hierarchy  (e.g.,  a  unit  is  in  danger 
of  not  meeting  mission  requirements).  These  design  characteristics  provide  feedback 
loops  from  which  the  commander  can  see  deviations  as  they  occur.  Feedback  loops  assist 
the  commander  with  making  decisions  that  help  influence  how  he  expects  events 
to  unfold. 


Figure  10.  Spatial  Synchronization  Display. 


(2)  Temporal  Synchronization  Display.  A  critical  aspect  of 
control  is  the  synchronization  of  combat  force  activities  toward  achieving  common  goals 
and  end-states.  Figure  1 1  depicts  a  representation  of  a  combat  activity  coordinating  tool 
(i.e.,  synchronization  matrix)  commonly  used  by  commanders  and  staff  personnel.  The 
synchronization  matrix  meshes  subordinate  unit  tasks  with  higher-level  purposes. 
Though  the  spatial  and  temporal  synchronization  displays  are  being  described  separately, 
it  is  important  to  realize  they  are  designed  to  work  together  in  a  complementary  fashion 
(Bennett  et  ah,  2008).  Essentially,  RAPTOR  nests  information  in  the  temporal 
synchronization  display  with  representations  located  in  the  spatial  synchronization 
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display.  This  nesting  technique  provides  the  commander  with  visually  salient  depictions 
of  critical  activities  to  be  coordinated  in  time  and  space. 

As  can  be  seen  in  Figure  11,  the  X  axis  of  the  display  depicts  time 
ranging  from  initiation  of  tactical  operations  to  X  +  projected  mission  completion  time. 
Present  time  for  the  engagement  is  depicted  by  the  thin  vertical  blue  line.  The  Y  axis 
captures  the  identities  of  the  maneuver  elements  represented  as  icons  in  the  spatial 
synchronization  display.  The  text  located  in  the  lighter  cell  areas  specify  tasks  and 
purposes  assigned  to  each  maneuver  unit.  The  thick  gray  vertical  lines  are  tied  to 
synchronization  points  presented  in  the  spatial  synchronization  display  (see  Figure  10) 
and  represent  preplanned  points  in  time,  or  conditions,  where  specific  unit  coordinating 
activities  must  be  accomplished. 
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figure  11.  Temporal  Synchronization  Display. 

(3)  Friendly  Combat  Resource  Display.  Combat  power  is 
perhaps  the  most  important  factor  a  commander  must  consider  when  planning  and 
executing  tactical  operations.  The  amount  and  type  of  resources  that  constitute  overall 
levels  of  combat  power  will  determine  the  feasibility  of  successfully  achieving  goals  and 
objectives.  As  discussed  in  the  opening  chapter,  combat  resources  are  finite,  and  their 
expenditure  must  be  proportional  to  any  advantages  gained  during  an  engagement.  Thus, 
the  commander  must  constantly  be  aware  of  the  levels  at  which  these  resources  exist,  and 
must  understand  how  their  expenditures  will  impact  tactical  operations. 

Since  overall  mission  accomplishment  hinges  on  the  ability  of 

subordinate  units  to  achieve  individual  assigned  tasks,  the  commander  must  constantly 
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know  resource  statuses  for  both  the  higher  and  intermediate  organizational  levels 
(i.e.,  battalion  [BN]  and  company  [CO]).  During  certain  situations,  the  commander  may 
also  need  to  know  lower-level  resource  statuses  (i.e.,  platoon  [PLT]  and  vehicle). 
Accordingly,  RAPTOR  provides  salient  representations  of  higher  and  intermediate 
combat  resource  levels.  RAPTOR  also  provides  the  commander  the  ability  to  access 
lower  combat  resource  levels  when  required. 

RAPTOR  displays  combat  resources  using  categorical,  analog,  and 
alphanumeric  designators  (see  Figure  12).  Alphabetical  designators  correspond  to 
specific  types  of  resources  (A:  ammunition,  F:  fuel,  P:  personnel,  T:  Ml  Abrams  tanks, 
B:  Bradley  Fighting  Vehicles).  Numeric  designators  correspond  to  quantities  of  a 
specific  resource  type  on  hand  (e.g.,  F/12907  shown  in  Figure  12  depicts  12,907  gallons 
of  fuel  on  hand).  The  color  coding  used  to  represent  statuses  are  in  accordance  with 
U.S.  Army  conventions  (green:  100%  -  85%;  amber:  84%  -  70%;  red:  69%  -  50%;  black: 
49%  and  lower). 


OigiOl  Values  Categorical 

(Raw  Count)  Boundaries 


Figure  12.  Friendly  Combat  Resource  Display. 

The  bars  in  the  resource  display  depict  analog  gauges 
corresponding  to  resource  percentages  on  hand.  These  bars  “shrink”  as  resources  are 
expended.  Bar  color  changes  match  percentage  thresholds  (e.g.,  bottom  amber  bar  in 
Figure  12  depicts  84%  -  70%  tanks  on  hand). 


38 


Categorical  conventions  depict  the  overall  color  for  the  least 
resource  percentage  on  hand.  For  example,  if  a  BN’s  aggregate  available  resource 
percentages  are  100%  (Green)  for  vehicles,  100%  (Green)  for  personnel,  84%  (Amber) 
for  ammunition,  and  69%  (Red)  for  fuel,  the  BN’s  categorical  color  convention  would  be 
red.  The  same  categorical  convention  is  also  applied  at  the  CO  level.  The  background 
color  for  both  the  combat  resource  display,  and  its  corresponding  combat  resource  icon 
represented  in  the  spatial  synchronization  display,  reflect  the  categorical  convention  for 
the  least  resource  percentage  on  hand  (see  “amber”  status  of  “Charlie”  CO  in  Figure  9). 

Though  resources  may  not  be  displayed  in  exact  quantities,  their 
representations  are  very  salient,  which  enables  the  commander  to  “spot  check”  and 
loosely  monitor  the  status  of  specific  resource  parameters  at  any  level  desired  (Bennett 
et  ah,  2008).  This  design  characteristic  provides  feedback  loops  from  which  the 
commander  can  anticipate  potential  deviations  before  they  occur,  and  to  make  decisions 
that  minimize  deviation  effects. 

(4)  Enemy  Combat  Resource  Display.  Commanders  and  staff 
personnel  need  infonnation  for  similar  types  of  combat  resources  and  capabilities  that  the 
enemy  possesses  in  order  to  anticipate  the  extent  to  which  the  enemy  can  impede  tactical 
operations.  In  reality,  determining  precise  types  and  quantities  of  enemy  resources  are 
rarely  achieved.  However,  major  enemy  combat  platforms  and  weapon  systems  can  be 
approximated  using  established  enemy  doctrinal  templates,  inputs  from  battlefield 
sensors,  and  various  forms  of  intelligence  estimates.  Commanders  and  staffs  use  these 
approximations  to  detennine  the  appropriate  allocation  of  friendly  resources  that  will 
enable  mission  success.  Just  as  the  commander  must  monitor  the  status  of  friendly 
resources  during  the  course  of  tactical  operations,  he  must  also  monitor  the  status  of 
enemy  resources  to  make  more  accurate  projections  for  how  he  anticipates  battlefield 
events  to  unfold. 

RAPTOR  provides  salient  representations  of  known  and  suspected 
enemy  resources  to  assist  the  commander  as  he  makes  decisions  based  on  projected 
outcomes  (see  Figure  13).  As  enemy  elements  are  positively  identified,  known 
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(i.e.,  alive)  enemy  values  increase,  and  suspected  (i.e.,  templated)  values  decrease  on  the 
display.  Consequently,  as  enemy  elements  are  disabled,  alive  values  decrease  while 
disabled  values  increase. 
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Figure  13.  Enemy  Resource  Display  (From:  Bennett,  Posey,  & 

Shattuck,  2008). 


Enemy  combat  resources  are  also  tied  to  representations  presented 
in  the  spatial  synchronization  display.  Their  specific  composition  and  strength  are 
represented  using  enemy  symbols  in  accordance  with  U.S.  Army  conventions  (see 
Figure  10).  Salient  visual  representations  of  enemy  combat  resources  assist  the 
commander  with  determining  how  to  employ  friendly  force  capabilities  based  on  the 
enemy’s  strengths  and  weaknesses. 

(5)  Force  Ratio  Display.  Another  extremely  important 
consideration  in  tactical  operations  is  the  relative  amount  of  combat  power  that  exists 
between  two  opposing  forces  at  any  point  in  time  (Bennett  et  al.,  2008).  Commanders 
and  staff  personnel  analyze  the  enemy’s  composition  (e.g.,  tank  and  infantry  fighting 
vehicle  quantities)  when  determining  appropriate  allocations  of  friendly  combat  resources 
that  will  enable  mission  success.  For  example,  U.S.  Army  doctrine  specifies  a  force  ratio 
of  at  least  three  to  one  when  attacking  enemy  defensive  positions. 
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Predetermined  estimates  for  how  force  ratios  are  anticipated  to 
evolve  must  be  continuously  monitored  during  an  engagement  to  assess  progress  toward 
achieving  mission  goals  and  objectives.  A  lack  of  progress  depicted  by  force  ratio 
changes  in  favor  of  the  enemy  will  drive  a  commander’s  decision  on  whether  to  continue, 
alter,  or  abort  mission  plans  (Bennett  et  ah,  2008). 

RAPTOR  provides  a  display  that  presents  the  commander  with 
salient  visual  representations  of  force  ratio  (see  Figure  14).  Two  trend  displays  are 
depicted  on  the  left  side  of  the  larger  force  ratio  display.  The  Y  axis  of  the  top  trend 
display  represents  planned  and  actual  values  of  friendly  force  ratios,  while  the  Y  axis  of 
the  bottom  trend  display  depicts  actual  and  planned  values  of  enemy  force  ratios.  The 
X  axis  of  both  trend  displays  depicts  time  ranging  from  the  engagement’s  initiation  to 
X  +  projected  mission  completion  time. 

Two  horizontal  bar  graphs  are  depicted  on  the  right  side  of  the 
larger  force  ratio  display.  The  top  horizontal  bar  graph  is  segmented  between  friendly 
tanks  and  Bradley  Fighting  Vehicles.  The  bottom  horizontal  bar  graph  is  also  segmented 
between  alive  and  templated  enemy  combat  platforms.  The  horizontal  extent  of  each  bar 
graph  toward  the  right  represents  the  force  equivalence  values  for  friendly  and  enemy 
forces  (Bennett  et  ah,  2008). 

The  two  horizontal  bar  graphs  are  connected  by  a  thick  blue  line 
(i.e.,  reflecting  line)  that  provides  a  visual  indicator  for  the  difference  between  friendly 
and  enemy  force  equivalence.  The  reflecting  line  also  intersects  the  force  ratio  values 
depicted  by  the  trend  displays.  The  line  expands  horizontally  across  these  displays  as 
time  progresses.  Favorable  friendly  force  ratios  are  represented  by  a  blue  line 
intersecting  through  the  top  trend  display.  Favorable  enemy  force  ratios  are  represented 
by  a  red  line  intersecting  through  the  bottom  trend  display.  Variations  (or  waves)  in  the 
reflecting  line  depict  rates  of  force  value  changes  over  time.  Dark  color  variations 
represent  actual  force  ratios,  while  light  color  variations  represent  planned  force  ratios. 
As  can  be  inferred,  reflecting  line  intersection  locations,  colors,  and  vertical  variation 
distances  provide  salient  feedback  loops  for  how  the  commander  can  expect  events 
to  unfold. 
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Figure  14.  Force  Ratio  Display  (From:  Bennett,  Posey,  &  Shattuck,  2008). 

(6)  Other  Perception  Enabling  Tools.  RAPTOR  also  provides 
additional  tools  designed  to  take  advantage  of  human  perception.  For  instance,  the  plan 
review  mode  is  a  tool  that  enables  commanders  to  track  how  an  engagement  was 
originally  planned  to  be  executed,  while  also  displaying  how  events  actually  progressed. 
A  simple  analogy  taken  from  the  analysis  of  a  football  play  makes  this  premise  concrete. 
As  a  television  football  commentator  analyzes  a  specific  play  for  the  audience,  he 
displays  a  visual  depiction  for  how  the  play  was  designed  to  be  executed  (e.g.,  arrows, 
lines,  Xs,  and  Os).  He  then  overlaps  a  recording  for  how  the  play  was  actually  executed 
so  the  audience  can  see  where  deviations  occurred. 

The  plan  review  mode  displays  planned  friendly  locations, 
activities,  and  resources  with  icons  containing  a  black  “X”  (actual  icons  do  not  contain  an 
X)  in  the  spatial  synchronization  display  (see  Figure  15).  The  plan  review  mode  also 
displays  and  highlights  preplanned  categorical  color  codes  on  the  right  side  of  the  analog 
bars  located  in  the  friendly  combat  resource  display.  The  plan  review  mode  enables  the 
commander  to  determine  exactly  which  deviations  occurred  in  combat  resource 
expenditures  during  precise  points  in  time. 
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Figure  15.  Plan  Review  Mode  (From:  Bennett,  Posey,  &  Shattuck,  2008). 


Another  tool  provided  by  RAPTOR  is  the  depiction  of  pre¬ 
planned,  alternative  COAs.  Commanders  and  staff  personnel  routinely  develop  multiple 
COAs  that  address  different  actions  subordinate  units  can  execute  based  on  likely,  or 
anticipated,  deviations  in  the  plan.  These  COAs  can  be  thought  of  as  “primed  plays”  the 
commander  can  “audible”  should  conditions  dictate  their  implementation.  The 
commander  can  review  preplanned,  alternative  COAs  at  any  time  by  rolling  the  cursor 
over  labeled  buttons  at  the  bottom  of  the  display  (see  Figure  9).  Representations  of  the 
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alternative  COAs  will  temporarily  replace  current  COA  representations  in  the  spatial  and 
temporal  synchronization  displays.  As  likely  deviations  in  the  current  COA  emerge,  the 
commander  can  choose  to  implement  a  preplanned,  alternative  COA  by  clicking  on  the 
corresponding  button,  then  verifying  the  change.  This  audible  constitutes  a  major 
modification  to  the  existing  plan,  which  can  be  viewed  by  other  friendly  elements 
monitoring  networked  RAPTOR  interfaces. 

b.  Direct  Manipulation 

For  an  interface  to  be  truly  effective  at  taking  advantage  of  human 
perceptual  capabilities  and  limitations,  the  displays  and  controls  of  the  interface  must  be 
designed  to  maintain  an  intact  perception-action  loop  (Talcott  et  ah,  2007).  Direct 
manipulation  is  a  critical  enabler  for  maintaining  this  loop  since  it  allows  operators  to  feel 
like  they  have  control  over  objects  located  in  the  environment.  An  analogy  using  vehicle 
operation  provides  a  concrete  example  for  how  direct  manipulation  is  tied  to  the 
perception-action  loop.  As  a  driver  operates  a  vehicle  in  the  environment,  he  perceives 
various  signals  (e.g.,  the  road,  other  vehicles,  pedestrians,  etc.)  that  prompt  actions  to  be 
executed.  As  the  driver  executes  actions  (e.g.,  manipulating  the  steering  wheel, 
accelerator,  and  brakes),  he  receives  immediate  feedback  (e.g.,  feels  vehicular  responses) 
for  how  well  the  executed  actions  enable  continued  vehicular  control.  Thus,  the 
fundamental  goal  in  achieving  direct  manipulation  is  to  allow  domain  practitioner’s  to 
execute  required  command  inputs  by  acting  directly  on  representations  in  the  interface 
(Bennett  et  ah,  2008). 

EID  principles  extend  the  benefits  of  direct  manipulation  to  interfaces 
located  in  complex  work  domains  (Vicente  &  Rasmussen,  1992).  The  objects  in  the 
interface  can  be  designed  to  support  both  perception  (direct  perception)  and  action  (direct 
manipulation).  When  this  symbiotic  relationship  exists  in  an  interface,  the 
perception-action  loop  is  intact.  RAPTOR  embraces  direct  manipulation  to  the  fullest 
extent  (Bennett  et  ah,  2008).  Unlike  FBCB2,  RAPTOR  does  not  use  command  lines  and 
pull-down  menus.  RAPTOR’S  design  allows  users  to  directly  act  on  what  they  see  in  the 
display  by  physically  manipulating  the  objects  on  their  screen  (Vicente  &  Rasmussen, 
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1992).  Thus,  all  potential  actions  by  the  commander  are  executed  directly  on  the 
interface.  The  merger  of  displays  and  controls  on  the  RAPTOR  interface  ensures  an 
intact  perception-action  loop,  thereby  enhancing  the  commander’s  perception  of  events 
and  actions  occurring  in  the  battlespace  (Bennett  et  ah,  2008).  The  following  sections 
describe  how  RAPTOR’S  design  enables  operators  to  directly  manipulate  representations 
located  in  the  interface. 

(1)  Synchronization  Points.  Synchronization  point 
representations  in  the  spatial  and  temporal  synchronization  displays  can  be  directly 
manipulated  to  assist  the  commander  with  controlling  friendly  force  activities  during 
tactical  operations  (see  Figures  10  and  11).  For  example,  the  commander  can  point, 
click,  drag,  and  release  synchronization  points  represented  in  the  spatial  synchronization 
display  to  change  destinations  where  a  unit  maneuvers  toward.  Similarly,  the  commander 
can  also  adjust  activity  timing  by  dragging  vertical  synchronization  points  represented  in 
the  temporal  synchronization  display.  Dragging  synchronization  point  lines  left  advances 
coordination  timing,  while  dragging  lines  right  delays  coordination  timing.  These 
manipulations  constitute  minor  adjustments  in  the  plan,  and  in  essence,  communicate  a 
command  directive.  Units  affected  by  the  change  are  able  to  view  the  modifications  as 
they  monitor  their  networked  interfaces. 

(2)  Graphical  Replay  Slider.  RAPTOR  provides  commanders 
with  the  ability  to  review  historical  events  and  preview  planned  events.  The  graphical 
replay  slider  is  located  at  the  top  of  the  temporal  synchronization  display  (see  Figure  9). 
The  graphical  replay  track  (i.e.,  horizontal  line)  represents  execution  time  ranging  from 
the  initiation  of  an  engagement  (i.e.,  extreme  left  limit)  to  X  +  projected  mission 
completion  time  (i.e.,  extreme  right  limit).  The  physical  location  of  the  manipulable 
slider  (i.e.,  square  button)  along  the  track  corresponds  to  current  time. 

The  commander  can  “rewind”  through  historical  events  by 
dragging  the  manipulable  slider  left.  He  can  then  view  a  “replay”  of  the  engagement  by 
dragging  the  slider  to  the  right.  The  replay  continues  until  the  commander  drags  the 
manipulable  slider  right  of  current  time,  which  then  changes  the  display  to  a  preview  of 
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preplanned  actions.  Current  mission  information  remains  displayed  as  the  commander 
reviews  and  previews  events.  The  manipulable  slider  springs  back  to  current  time 
when  released. 

Thus,  the  graphical  replay  slider  enables  the  commander  to  see 
exactly  which  deviations  occurred  during  specific  points  in  time.  The  slider  also  enables 
the  commander  to  anticipate  potential  future  deviations  as  he  previews  planned  activities. 
This  design  feature  provides  feedback  loops  that  assist  the  commander  in  determining  the 
extent  to  which  preplanned  actions  will  enable  projected  events  to  unfold  as  expected. 

(3)  Other  Manipulable  Tools.  As  stated  earlier,  higher  levels 
of  representation  enable  operators  to  better  cope  with  complex  and  novel  situations. 
However,  effective  interface  designs  support  both  intuitive  and  analytical  modes  of 
thinking  by  enabling  the  operator  to  choose  which  level  of  aggregation  to  view 
information  from.  Accordingly,  RAPTOR  provides  several  options  that  allow  the 
operator  to  access  and  track  several  levels  of  information  in  an  efficient  and  economical 
fashion. 

The  tree  control  and  button  controls  located  on  the  top  right-hand 
side  of  the  interface  provide  options  for  selecting  various  levels  of  combat  resource 
displays  (see  Figure  16).  The  tree  control  is  similar  to  task-organization  charts 
commonly  used  by  commanders  and  staff  personnel,  and  depicts  the  overall  unit  structure 
down  to  the  CO,  PLT,  and  vehicle  levels.  The  tree  control  provides  a  mechanism  for  the 
commander  to  view  combat  resources  at  any  level  desired.  The  default  setting  depicts 
combat  resource  displays  at  higher  (BN)  and  intermediate  (CO)  levels  (see  Figure  9). 
The  “bubble”  color  codes  represent  categorical  combat  resource  statuses. 

The  commander  can  temporarily  change  the  viewable  level  by 
rolling  the  cursor  over  any  element  depicted  in  the  tree  control.  For  example,  rolling  the 
cursor  over  A  CO  will  highlight  the  “A”  bubble,  and  will  change  the  higher-level  combat 
resource  display  to  A  CO.  This  will  also  change  the  intermediate-level  resource  display 
to  the  three  PLTs  assigned  to  A  CO.  Similarly,  the  commander  can  continue  to  access 
combat  resource  information  down  to  the  vehicle  and  individual  crew  levels  by  rolling 
the  cursor  over  a  PLT  (e.g.,  Al)  or  individual  vehicle  bubble.  Conversely,  the 
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commander  can  select  a  permanent  view  by  pointing  and  clicking  on  a  desired  bubble. 
The  combat  resource  display  changes  back  to  the  default  settings  once  the  commander 
either  removes  the  cursor  from  the  lower  level  bubbles,  or  clicks  the  BN  bubble. 


Control  Buttons  for 
Selecting  Combat  Resource 
Icons  On  Map 


Control  Tree  for 
Selecting  Combat 
Resource  Displays 


Figure  16.  Tree  Control  &  Button  Controls. 


The  control  buttons  located  on  the  left  side  of  the  tree  control  (see 
Figure  16)  enable  the  commander  to  view  physical  locations  of  higher-,  intermediate-, 
and  lower-level  combat  resource  icons  located  in  the  spatial  synchronization  display  (see 
Figure  9).  RAPTOR’S  default  setting  represents  combat  resource  icons  at  the  CO  level. 
As  stated  previously,  the  icon  color  codes  correspond  to  categorical  combat  resource 
statuses.  The  commander  can  choose  to  temporarily  view  all  combat  resource  icons  at 
higher  (i.e.,  BN)  or  lower  levels  (i.e.,  PLT  and  vehicle)  by  rolling  the  cursor  over  the  BN, 
PLT,  or  vehicle  control  buttons.  Conversely,  the  commander  can  select  a  permanent  icon 
level  by  pointing  and  clicking  on  a  desired  button.  The  default  setting  is  reestablished 
once  the  commander  either  removes  the  cursor  from  a  button,  or  clicks  the  CO  button. 

Viewing  the  entire  range  of  combat  resource  icons  at  lower  levels 
clutters  the  spatial  synchronization  display  and  provides  potentially  overwhelming 
amounts  of  information.  Therefore,  RAPTOR’S  design  enables  the  commander  to  access 
any  combination  of  lower-level  unit  information  in  an  easy  and  efficient  manner.  For 
example,  the  commander  can  display  a  company’s  lower-level  resource  icons  by  pointing 
and  clicking  on  a  specific  CO  icon  in  the  spatial  synchronization  display.  This  results  in 
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the  replacement  of  that  CO  icon  with  its  three  PLT  icons.  The  other  CO  icons  remain 
unchanged.  Similarly,  vehicle  level  icons  can  be  displayed  by  clicking  a  PLT  icon.  This 
design  feature  enables  a  commander  to  successively  drill  down  to  desired  levels  and 
views.  Default  settings  are  reestablished  by  clicking  the  CO  button  located  in  the  button 
controls  (see  Figure  16). 

Finally,  the  commander  can  also  “magnify”  portions  of  the 
topographical  map  presented  in  the  spatial  synchronization  display.  As  previously 
discussed  (see  Chapter  I.A.),  terrain  posses  several  constraints  that  potentially  impede 
progress  during  tactical  operations.  Thus,  the  commander  must  determine  the  effects 
terrain  will  have  on  friendly  resources  as  they  execute  tasks.  Screen  resolution  settings, 
and  multiple  representations  arrayed  on  the  map  often  masks  terrain  contour  lines.  The 
commander  can  investigate  finer  terrain  details  by  pointing  on  the  purple-colored  reticule, 
and  turning  the  mouse  wheel  clockwise  (see  Figure  10).  Selected  portions  of  the  map 
will  magnify  to  higher  resolution  levels.  Turning  the  wheel  counterclockwise  restores  the 
selected  portion  to  the  default  resolution  settings. 

E.  PREVIOUS  RAPTOR  STUDIES  AND  EVALUATIONS 

RAPTOR  has  already  demonstrated  its  potential  at  improving  the  military 
practitioner’s  performance  while  executing  C  of  tactical  scenarios.  During  a  study  in 
2007,  active  duty  U.S.  Army  officers  were  required  to  perform  well-constrained,  but 
critical  tasks  of  obtaining  friendly  force  combat  resource  information  (e.g.,  fuel  and 
ammunition)  at  three  different  echelon  levels  (Talcott  et  al.,  2007).  Participants 
performed  the  study  in  a  controlled  laboratory  setting  using  simulations  of  both  the 
FBCB2  and  RAPTOR  interfaces.  The  results  of  the  study  showed  that  RAPTOR  was 
superior  to  FBCB2  in  all  assessment  categories  (quantitative,  categorical,  and  needs), 
dependent  variables  (accuracy,  latency),  and  echelon  levels  (BN,  CO,  PLT).  The 
conclusion  of  the  study  determined  that  perception-icon  design  strategy  was  very 
effective  in  that  experimental  context.  Actual  or  potential  applications  from  the  study 
included  both  specific  interface  design  strategies  for  military  C  and  general  interface 
design  principles  for  intermediate  work  domains  (Talcott  et  al.,  2007). 
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Another  study  was  conducted  to  investigate  aspects  of  decision  making  (Bennett 
et  al.,  2008).  Two  versions  of  RAPTOR  (i.e.,  enhanced  and  baseline)  were  developed  for 
the  study.  Active  duty  U.S.  Army  officers  assumed  the  role  of  BN  commander  and 
viewed  dynamic,  authentic  tactical  scenarios  (either  offensive  or  defensive)  using  one  of 
the  two  interfaces.  The  participants  were  required  to  answer  specific  questions  pertaining 
to  the  scenario  at  six  different  points  that  coincided  with  critical  events.  The  results  of 
the  study  showed  that  those  participants  who  used  the  enhanced  RAPTOR  interface 
exhibited  a  greater  tendency  to  produce  references  to  plans  and  operations  orders.  Twice 
as  many  references  to  mission  plans  were  made  by  those  participants  using  the  enhanced 
interface  (52)  than  those  using  the  baseline  interface  (26).  Substantially  more  references 
to  the  mission  operations  order  were  also  made  by  participants  using  the  enhanced 
version  (24  versus  15)  (Bennett  et  al.,  2008). 

RAPTOR’S  initial  successes  illustrate  the  interface’s  potential  to  facilitate  better 
decision-making  and  enhanced  SA  as  military  practitioners  C  tactical  operations.  Thus, 
this  study  aims  to  further  advance  the  development  of  RAPTOR  by  building  on  work 
previously  conducted.  This  study  will  also  aims  to  validate  the  interface’s  ability  to 
increase  total  system  performance  as  users  deal  with  uncertainty  and  novel  situations 
inherent  in  dynamic  and  fluid  environments. 

F.  HYPOTHESES 

The  literature  review  has  uncovered  many  important  questions  concerning  the 
ability  of  interfaces  to  increase  human  performance  during  C  .  Though  much  of  the 
concepts  previously  described  yield  multiple  interesting  topics  that  could  be  explored  in 
considerable  depths,  the  most  relevant  questions  have  been  narrowed  to  those  pertaining 
to  this  study’s  specific  research  objectives.  Accordingly,  the  alternative  hypotheses 
generated  from  those  questions  are  as  follows: 

•  Hap  The  RAPTOR  interface  leads  to  better  levels  of  SA  than  the 
U.S.  Army’s  FBCB2  interface. 

•  Haa:  The  RAPTOR  interface  supports  better  decision-making  processes 
than  the  U.S.  Army’s  FBCB2  interface. 
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Ha3:  The  RAPTOR  interface  requires  less  cognitive  workload  than  the 
U.S.  Army’s  FBCB2  interface. 


III.  METHOD 


A.  EMPIRICAL  STUDY  OVERVIEW 

1.  Research  Design 

A  controlled  laboratory  experiment  was  used  to  assess  military  decision-maker 
performance  while  performing  critical  battlefield  activities  (e.g.,  acquiring  and  analyzing 
critical  knowledge  on  the  effects  of  terrain;  assessing  anticipated  enemy  actions  on 
friendly  force  operations).  This  study  was  a  2  x  2  factorial  mixed  subjects  design 
comparing  two  interfaces  (RAPTOR  and  Baseline)  and  two  tactical  scenarios  (attack  and 
raid).  Participants  were  randomly  assigned  to  four  groups  (RAPTOR  Group  1,  RAPTOR 
Group  2,  Baseline  Group  1,  and  Baseline  Group  2).  Each  group  conducted  both  tactical 
scenarios  using  only  one  type  of  interface  (i.e.,  groups  were  blocked  against  one  type  of 
interface).  The  Tactical  scenario-interface  combinations  were  counterbalanced  to  control 
for  an  order  effect.  Figure  17  illustrates  the  design  used  for  this  study. 


2x2  Mixed  Design 

SCENARIOS 

Attack 

Raid 

in 

RAPTOR  1 

Trial  1 

Trial  2 

o 

RAPTOR  2 

Trial  2 

Trial  1 

o 

— 

Baseline  1 

Trial  2 

Trial  1 

CO 

Baseline  2 

Trial  1 

Trial  2 

Figure  17.  Research  Design  Example. 


Participants  were  shown  U.S.  Army  BN-level  tactical  operation  displays  driven 
by  an  interactive  simulation  technology,  the  Distributed  Dynamic  Decision-Making 
(DDD  4.0).  Each  tactical  scenario  lasted  25  minutes  in  duration.  Participants  played  the 
role  of  a  BN  commander,  and  performed  various  activities  associated  with  the  C  of 
multiple  manuever  elements.  Numerous  measurements  were  collected  to  gain  insight 
into  participant  decision-making  processes,  SA,  and  cognitive  workload  as  they 
progressed  through  the  scenarios. 
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2.  Study  Approach 

As  previously  stated,  when  determining  an  interface’s  effectiveness  at  assisting 
the  warfighter  with  maintaining  SA  and  decision-making  processes,  one  must  employ  a 
model  that  includes  both  human  and  machine.  Accordingly,  this  study  used  the  DMSC 
(Miller  &  Shattuck,  2006)  as  the  theoretical  framework  to  determine  if  RAPTOR 
enhanced  human  performance  during  the  C  of  tactical  operations.  Furthermore,  this 
study  also  proposes  the  Tactical  Rating  of  Awareness  for  Combat  Environments 
(TRACE)  tool  (see  Figure  18)  as  an  evaluation  strategy  to  determine  levels  of  SA  for  the 
participants. 

The  DMSC  model  is  composed  of  six  ovals.  The  first  three  ovals  represent 
technological  contributions  to  the  model,  while  the  remaining  three  ovals  represent 
human  contributions  to  the  model.  Thus,  data  were  captured  for  all  six  ovals  during  the 
study.  Data  from  the  RAPTOR  interface  representations  and  the  DDD  4.0  simulation 
technology  populated  the  first  three  ovals,  while  TRACE,  CCIR,  and  critical  event 
measures  captured  human  data  for  the  last  three  ovals.  Table  1  lists  the  measures  used  to 
populate  the  ovals. 


Table  1.  Measures  For  Populating  DMSC  Ovals  (After:  Read,  2007). 


Oval 

Lab  Experiment 

1 

ODD  4  0  Interactive  Simulation  Technoloqv 
API  Intelligence  Agent  Algorithms 

Network  Data  Storage 

2 

Combat  Resource  Parameters 

Combat  Resource  Capability  Values 

Sensor  Capability  Values 

Probabilities  Hit/Kill 

3 

Data  Representations 

Displays  and  Other  Graphical  Screen  Shots 
Interface  Menus.  Tools,  and  Options 

V 

4.5.6 

TRACE  Quenes 

Commander's  Critical  Info  Requirements 
Critical  Event  Critena 

Audio  and  Video  Recordinq  Devices 

3.  Independent  Variables 

•  Interface  Type  -  RAPTOR  and  Baseline. 

•  Tactical  Scenarios  -  attack  and  raid. 
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4.  Dependent  Variables 

•  TRACE  Scores  -  accuracy  of  participant  responses  to  periodic  situation 
reports  (SITREP)  initiated  by  researchers  to  query  for  levels  of  SA. 

•  TRACE  Latency  -  the  elapsed  time  from  when  the  researcher  requests  the 
participant  to  send  a  SITREP  to  when  the  participant  answers  all  line 
item  entries. 

•  Critical  Event  Latency  -  the  elapsed  time  from  when  decision  point 
criteria  are  met  to  when  the  participant  announces  decision  point  criteria 
have  been  met. 

•  Critical  Information  Latency  -  the  elapsed  time  from  when  a  commander’s 
critical  information  requirement  (CCIR)  is  available  to  when  the 
participant  reports  the  CCIR  answer. 

•  Critical  Infonnation  Scores  -  accuracy  of  participant  responses  to 
CCIR  questions. 

•  Continuous  Subjective  Workload  Assessment  Technique  -  participant 

periodic  entry  of  self-reported  workload  throughout  the  tactical  scenarios. 

•  Total  Requests  for  Infonnation  -  total  number  of  times  a  participant  refers 
to  an  operation  order  during  an  entire  scenario. 

5.  Study  Setting 

Data  collection  occurred  at  the  Naval  Postgraduate  School  (NPS)  in  Monterey, 
California.  NPS  contains  a  large  pool  of  available  U.S.  Army  Officers  assigned  to 
Maneuver,  Fires,  and  Effects  (MFE)  basic  branches.  Officers  assigned  to  MFE  branches 
can  be  considered  as  experienced  practitioners  of  C  since  they  receive  professional 

education  and  extensive  training  on  how  to  effectively  operate  the  C  system,  and  also 

2 

routinely  perform  C  activities  during  tactical  operations. 

6.  Participants 

Participants  consisted  of  16  male  U.S.  Army  Officers  with  an  average  age  of  36.8 
years.  Fourteen  participants  held  the  rank  of  0-4,  and  two  held  the  rank  of  0-5.  Fifteen 
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Officers  had  combat  experience  in  either  Iraq  or  Afghanistan,  with  an  average  time  of  1 5 
months  in  combat  (SD  =  8.6).  Eight  participants  had  operational  experience  with  military 
operations  other  than  war  (MOOTW),  with  an  average  time  of  4  months  (SD  =  4.3).  The 
average  number  of  deployments  to  combat  zones  and  MOOTW  was  1  (SD  =  8.8). 
Fifteen  participants  conducted  mission  rehearsal  exercises  at  a  Combat  Training  Center 
(CTC).  The  average  number  of  rotations  to  a  CTC  was  3  (SD  =  1.8).  Twelve 
participants  had  previous  experience  using  a  C  interface  (i.e.,  FBCB2)  either  during 
tactical  exercises  or  combat  operations.  Participants  had  no  previous  experience  with 
either  the  RAPTOR  interface  or  the  Baseline  interface.  All  participants  had  previous 
command  experience,  with  an  average  of  29  months  time  in  command  (SD  =  1 1.0).  All 
participants  had  nonnal  (or  corrected)  visual  acuity  and  color  perception.  Participants 
were  not  given  monetary  compensation  for  their  participation. 

B.  APPARATUS 

1,  Instrumentation 

All  experimental  events  were  controlled  by  identical  computers  (Dell  Precision 
M6300  laptops,  Fimerick,  Ireland,  777  MHz),  with  identical  color  displays  (Dell 
Computer,  Fimerick,  Ireland,  UltraSharp,  17”,  1920  X  1200  resolution,  Model  WUXGA) 
with  built-in  standard  QWERTY  keyboards  and  Dell  2-Button  USB  Optical  Mouse  with 
scrolling  wheel. 

As  stated  earlier,  tactical  scenarios  were  presented  to  participants  using  the  DDD 
4.0  internet  based  simulation  technology.  Simulations  were  controlled  through  Aptima 
Inc.’s  interactive  client  server.  Scenario  environmental  conditions,  friendly,  and  enemy 
activities  were  controlled  by  DDD  4.0’s  Agent  Application  Program  Interface  (API) 
intelligence  algorithms.  These  algorithms  standardized  discrete  activities  to  ensure 
participants  encountered  the  same  events  during  all  scenarios. 

Digital  audio  and  video  recording  devices  were  used  to  collect  participant  data 
during  all  experimental  events.  Video  recording  devices  consisted  of  3  communication 
cameras  (Canon,  Oita,  Japan,  16x  zoom,  440,000  effective  pixels,  47.5°  to  3°  view  angle, 
4.0  to  64.0mm  minimum  focus,  Model  VC-C4)  that  transmitted  recorded  video  images 
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directly  to  researcher  laptops  via  Cat  5  Ethernet  connections.  Audio  recording  devices 
consisted  of  portable  digital  voice  recorders  (Olympus,  Tokyo,  Japan,  WMA  recording 
format,  -70  dBv  input  level,  44.1  kHz  sampling  frequency,  Model  DS-50).  Recorded 
video  data  were  transferred  to  DVDs,  and  recorded  audio  file  MP3  data  were  transferred 
onto  researcher  laptops  for  further  analysis. 

Participants  were  also  provided  with  hard  copies  of  the  tactical  operations  orders, 
blank  copies  of  the  TRACE  tool,  a  copy  of  the  CCIR  reporting  fonnat,  a  copy  of  the 
critical  event  reporting  format,  additional  notepaper,  a  pen,  and  a  calculator.  Researchers 
collected  completed  TRACE  tools  and  reporting  formats  at  the  conclusion  of  each 
experimental  event  for  further  analysis. 

2.  Materials 

a.  TRACE  Tool 

Since  military  practitioners  are  accustomed  to  gathering  and  disseminating 
friendly  and  enemy  force  assessments  via  situation  report  (SITREP)  during  tactical 
operations,  the  TRACE  tool  was  developed  to  provide  researchers  with  a  method  for 
minimizing  obtrusive  data  collection.  The  TRACE  tool  is  flexible  enough  to  be  applied 
to  various  types  of  experiments  or  training  (simulated  and/or  field  settings)  that  aim  to 
measure  levels  of  military  practitioner  SA  during  tactical  situations.  It  was  developed 
using  different  U.S.  Army  reports  listed  in  FM  101-5-2  (1999).  These  reports  were 
refined  for  data  collection  purposes,  and  combined  into  a  standard  U.S.  Army  SITREP 
format  (see  Figure  18). 
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Figure  18.  TRACE  Tool  Overview. 

The  TRACE  tool  was  used  to  populate  Ovals  4  to  6  of  the  DMSC  with 
human  SA  data.  TRACE  measurements  include  timed  responses  to  queries  for  specific 
information,  and  the  accuracy  of  the  information  provided.  As  can  be  seen  in  Figure  18, 
queries  for  numerous  types  of  friendly  and  enemy  information  are  grouped  under  seven 
specific  line  item  entries.  This  format  provides  participants  with  a  logical  sequence  for 
reporting  key  pieces  of  information  as  they  attempt  to  make  sense  of  the  battlefield 
situation.  Lines  1-3  pertain  to  the  historical  activities  of  individual  friendly  and  enemy 
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elements  on  the  battlefield.  Participant  responses  to  these  queries  provide  infonnation  on 
Level  1  SA  for  populating  Oval  4  (Perception).  Lines  4  to  6  pertain  to  the  current  status 
of  friendly  and  enemy  capabilities.  Participant  responses  to  these  queries  provide 
information  on  Level  2  SA  for  populating  Oval  5  (Comprehension).  Line  7  pertains  to 
immediate  future  friendly  actions  based  on  the  overall  battlefield  situation.  Participant 
responses  to  these  queries  provide  infonnation  on  Level  3  SA  for  populating 
Oval  6  (Projection). 

To  maintain  consistent  temporal  references,  TRACE  measures  were  synchronized 
to  simulation  time  and  captured  by  the  laptop  computers  presenting  the  simulations. 
TRACE  measures  were  also  captured  by  video  recording  devices  focused  on  interface 
screens.  Participants  annotated  their  responses  to  line  item  entry  queries  onto  hard  copy 
TRACE  tools,  while  their  voice  responses  to  the  queries  were  captured  by  audio 
recording  devices. 

b.  Commander’s  Critical  Information  Requirements  (CCIR) 

CCIR  is  a  comprehensive  list  of  information  requirements  identified  by 
the  commander  to  facilitate  timely  IM  and  the  decision-making  process  that  affect 
successful  mission  accomplishment  (Department  of  the  Army,  2004).  CCIR  is 
essentially  a  list  of  questions  pertaining  to  enemy  activities,  friendly  activities,  and  the 
environment  that  must  be  answered  to  enable  the  commander  to  maintain  SA,  project 
future  activities,  and  make  timely  decisions  (Department  of  the  Army,  2003).  As  such, 
CCIR  is  normally  comprised  of  two  key  subcomponents:  priority  intelligence 
requirements  (PIR)  and  friendly  force  information  requirements  (FFIR). 

Specific  CCIRs  (containing  both  PIRs  and  FFIRs)  pertaining  to  each 
tactical  scenario  were  provided  to  the  participants  (see  Figures  19  and  20).  As 
highlighted  previously,  scenario  environmental  conditions,  friendly,  and  enemy  activities 
were  controlled  by  DDD  4.0’s  API  intelligence  algorithms  which  populated  DMSC  Ovals 
1-2.  Furthermore,  these  conditions  and  activities  were  presented  to  participants  as 
graphical  representations  in  the  various  interface  displays  which  populated  Oval  3. 
Participant  temporal  recognition  of  CCIR  activities  as  they  transpired  on  the  battlefield 
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were  used  to  further  populate  Oval  4  (perception).  Participant  responses  to  CCIR  queries 
were  used  to  further  populate  Oval  5  (comprehension). 
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Figure  19.  CCIR  List  and  Report  Format  for  Attack  Scenario. 
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COMMANDER  CRITICAL  INFORMATION  REPORT  (CCIR)  (Raid  Scenario) 
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Figure  20.  CCIR  List  and  Report  Format  for  Raid  Scenario. 

To  maintain  consistent  temporal  references,  CCIR  events  were 
synchronized  to  simulation  time,  and  were  captured  by  both  the  laptop  computers 
presenting  the  simulations  and  by  video  recording  devices.  Participants  annotated  CCIR 
queries  onto  CCIR  reporting  formats,  and  their  voice  responses  were  captured  by  audio 
recording  devices. 


c.  Decision  Points 

As  discussed  earlier,  FM  6-0  (2003)  describes  decision  making  as 
selecting  the  one  most  favorable  COA  to  accomplish  a  mission.  Therefore,  participants 
were  provided  three  preplanned  COAs  and  were  expected  to  select  one  of  them,  based  on 
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their  comprehension  of  the  friendly  and  enemy  force  situation  and  activities  (see  Section 
D.I.).  The  decision  to  choose  a  specific  COA  was  tied  to  critical  event  criteria  listed  in 
decision  support  matrixes  that  were  provided  to  the  participants  as  an  annex  in  the  tactical 
OPORDs  (see  Appendices  A  and  B). 

The  point  in  time  and  space  in  which  a  participant  was  required  to  execute 
a  specific  COA  was  represented  by  a  graphical  control  symbol  known  as  a  decision  point 
(DP)  (see  below  Section  C.I.).  FM  101-5  (1997)  describes  DPs  as  critical  events  or 
locations  on  the  battlefield  where  tactical  decisions  are  required  during  mission 
execution.  However,  DPs  do  not  dictate  the  decision  to  be  made,  only  that  a  decision 
must  be  made,  as  well  as  when  and  where  it  should  be  made  in  order  to  have  the 
maximum  impact  on  friendly  and/or  enemy  CO  As. 

Consequently,  the  availability  of  multiple  COAs  whose  selection  are 
dependent  upon  a  participant’s  perception  of  critical  events  (or  cues)  represented  by 
technological  agents  in  a  time  constrained  and  uncertain  environment  is  consistent  with 
the  NDM  characteristics  taken  into  account  by  the  DMSC  model  (see  Chapter  H.3.C.). 
Critical  events  were  controlled  by  DDD  4.0’s  API  intelligence  algorithms  which 
populated  Ovals  1  to  2,  and  were  presented  to  participants  as  graphical  representations  in 
the  various  interface  displays  which  populated  Oval  3.  Participant  temporal  recognition 
of  these  critical  events  were  used  to  further  populate  Oval  4  (Perception),  while  their 
selection  of  a  specific  COA  was  used  to  further  populate  Oval  5  (Comprehension)  and 
Oval  6  (Projection).  Finally,  COA  selections  provided  insights  for  how  participants 
expected  events  to  unfold,  and  the  accuracy  of  these  decisions  was  represented  by 
feedback  loops  from  Oval  6  to  Oval  2. 

To  maintain  consistent  temporal  references,  critical  events  were 
synchronized  to  simulation  time,  and  were  captured  by  both  the  laptop  computers 
presenting  the  simulations  and  by  video  recording  devices  as  well.  Participants  annotated 
critical  event  queries  onto  DP  reporting  formats,  and  their  voice  responses  were  captured 
by  audio  recording  devices. 
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d. 


Workload 


Adams  et  al.  (1995)  argues  that  acquiring  and  maintaining  high  levels  of 
SA  must  be  appreciated  as  an  integral  part  of  the  operator’s  mental  workload.  Wickens 
(2008)  further  explains  that  an  increase  in  workload  can  divert  scarce  cognitive  resources 
from  maintaining  SA,  while  a  well-designed  usable  display  can  both  reduce  workload 
and  increase  SA.  However,  Van  Orden  (2001)  also  suggests  that  a  reduction  in  workload 
can  lead  to  complacent  behavior  caused  by  increased  operator  reliance  on  technology  and 
automation,  which  can  also  result  in  a  loss  of  SA. 

Accordingly,  an  estimation  of  workload  commonly  referred  to  as  the 
continuous  subjective  workload  assessment  technique  (C-SWAT)  was  incorporated  to 
elicit  participant  perceived  levels  of  workload  during  the  experiment.  The  C-SWAT 
utilized  is  a  simple,  non-obtrusive  workload  estimation  technique  borrowed  from 
previous  workload  studies  conducted  by  Van  Orden  (2001).  Like  those  studies,  the  7 
-point  scale  used  in  this  study  was  also  anchored  only  by  the  descriptors  shown  in 
Figure  2 1 .  The  workload  estimation  appeared  on  the  top  portion  of  participant  displays 
every  5  minutes  during  the  25-minute  tactical  scenarios.  Audio  prompts  were 
incorporated  to  alert  participants  of  the  workload  estimate’s  appearance.  Participants 
entered  their  perceived  level  of  cognitive  workload  once  prompted. 


Figure  2 1 .  Subjective  Workload  Estimation  Prompt. 

Researchers  also  annotated  the  number  of  times  participants  were  forced 
to  divide  their  attention  between  monitoring  interface  displays  and  accessing  additional 
information  located  in  hard  copy  tactical  OPORDs  as  another  simple  workload  measure. 
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C.  INTERFACES 

Interface  displays,  menus,  tools,  options,  data  representations,  and  other  various 
graphical  screen  shots  described  previously  in  Chapter  II. D  (RAPTOR  design),  and 
during  the  following  description  of  the  baseline  interface,  were  used  to  populate 
DMSC  Oval  3. 

1.  RAPTOR  Interface 

Chapter  2  provides  detailed  explanations  of  the  displays  and  functions  associated 
with  the  RAPTOR  interface.  Participants  using  RAPTOR  were  pennitted  to  use  most  of 
the  interface  tools  and  options  previously  described,  and  to  incorporate  data  provided  by 
the  various  displays  as  they  executed  C  activities  during  the  tactical  simulations. 
However,  participants  were  unable  to  manipulate  synchronization  points  represented  in 
the  spatial  and  temporal  synchronization  displays.  Participants  were  also  unable  to 
manipulate  the  “Preview”  mode  to  view  pre-planned  actions  in  time  and  space,  but  were 
able  to  access  the  “Review”  mode  to  view  historical  activities.  Additionally,  friendly 
combat  resource  icons  were  held  constant  at  the  company  level  hierarchy,  and  the 
magnification  reticule  was  disabled.  Holding  these  tools  constant  standardized  friendly 
force  actions  across  each  simulation,  which  ultimately  enabled  researchers  to  better 
control  for  unanticipated  outcomes  and  to  gather  more  meaningful  measurements. 

2.  Baseline  Interface 

An  alternative  interface  (Baseline)  was  developed  in  order  to  compare  participant 
performance  with  the  RAPTOR  interface.  The  baseline  interface  was  modeled  after  the 
U.S.  Anny’s  FBCB2  interface.  Although  the  baseline  interface’s  appearance,  displays, 
tools,  and  options  are  not  exactly  the  same  as  those  provided  by  FBCB2,  their 
functionality  and  informational  structures  are  comparable  (see  Figure  22).  For  instance, 
like  FBCB2,  the  baseline  interface  requires  users  to  operate  various  command  lines  and 
pull-down  menus  to  access  different  types  and  levels  of  data.  Also  like  FBCB2,  much  of 
the  data  represented  by  the  baseline  interface  are  presented  in  alphanumeric 
(i.e.,  text)  formats. 
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Figure  22.  Baseline  Interface  (main  screen). 


A  more  concrete  example  of  these  similarities  is  illustrated  by  the  activity 
sequence  required  to  obtain  combat  resource  values  (e.g.,  percent  of  available  fuel)  at  the 
BN  level  when  using  the  baseline  interface.  This  sequence  is  analogous  to  the  activity 
sequence  required  when  using  FBCB2  as  described  by  Talcott  et  al.  (2007).  Combat 
resource  data  can  be  obtained  by  activating  the  “FIPR”  reports  button  located  on  the  right 
side  of  the  screen  (see  Figure  22).  The  button  is  clicked  to  access  the  “FIPR”  menu, 
which  provides  a  series  of  reports  categorized  by  precedence  and  “filed”  under  flash, 
immediate,  priority,  and  routine  (FIPR)  tabs.  Combat  resource  reports  (i.e.,  LOGSTAT 
Reports)  are  accessed  by  clicking  on  the  “Routine”  tab  (see  Figure  23). 
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Figure  23.  FIPR  Menu  for  Baseline  Interface. 


Detailed  combat  resource  data  can  be  obtained  by  activating  a  Company  level 
report  (e.g.,  CO  B)  listed  under  the  routine  tab,  which  produces  a  pre-formatted  report 
screen  containing  an  alphanumeric  data  sheet  (see  Figure  24).  The  desired  parameter 
value  must  be  located  within  the  alphanumeric  data  and  either  manually  recorded  or 
remembered.  This  process  must  then  be  repeated  for  each  Company  element  within  the 
Battalion  level  organization,  followed  by  the  computation  of  the  aggregate  parameter 
value  (either  manually  or  mentally). 
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Figure  24.  Logistical  (LOGSTAT)  Report  Example  (e.g.,  CO  B). 


64 


FBCB2  enables  users  to  generate  and  send  numerous  types  of  reports  and  “long 
form”  combat  messages  (i.e.,  e-mails)  to  multiple  platforms  networked  within  the  tactical 
internet.  FBCB2  users  can  also  determine  the  level  of  precedence  for  each  report  and 
combat  message  they  send.  However,  according  to  Talcott  et  al.  (2007),  field  studies  of 
the  FBCB2  interface  conducted  by  the  Center  for  Army  Lessons  Learned  (CALL) 
indicate  that  commanders  and  their  staffs  tend  to  become  inundated  by  the  amount  of  data 
presented  and  the  amount  of  effort  required  to  interpret  these  data.  This  is  particularly 
true  during  combat  situations  when  high  stress  and  heavy  workloads  are  imposed. 
Therefore,  for  simplicity  purposes,  the  baseline  interface  in  the  present  study  provides 
participants  with  only  four  types  of  tactical  reports  “generated”  by  subordinate  units. 
Furthermore,  each  of  these  reports  are  generated  based  on  predetermined  events, 
categorized  by  fixed  levels  of  precedence,  organized  by  unit  and  reporting  time,  and 
displayed  in  standardized  formats  to  provide  participants  with  a  more  efficient  process  for 
selecting,  interpreting,  and  integrating  the  data  provided.  Figure  25  illustrates  the  fixed 
levels  of  precedence  categories  and  the  predetennined  generating  events  for  each  tactical 
report.  Figure  26  illustrates  the  standardized  alphanumeric  formats  used  for  each 
tactical  report. 


Report  Type 

Precedence  Category 

Generation  Event 

Contact  Report 

Flash 

Only  during  the  initial  contact 
with  an  enemy  element 

Spot  Report  (SPOTREP) 

Immediate 

As  new  enemy  forces  are 
identified 

Enemy  Battle  Damage 
Assessment  Report  (E-BDA) 

Priority 

As  enemy  combat  resources 
are  destroyed 

Logistics  Report  (LOGSTAT) 

Routine 

As  company  level  combat 
resources  change 

Figure  25.  Tactical  Report  Methodology. 
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Figure  26.  Standardized  Report  Formats  for  the  Baseline  Interface. 


The  RAPTOR  and  baseline  interface  designs  are  substantially  different,  but  their 
informational  content  is  equivalent.  In  other  words,  much  of  the  data  and  hierarchical 
relationships  represented  by  the  RAPTOR  interface  are  also  represented  by  the  baseline 
interface,  though  these  representations  are  presented  in  different  formats  and  styles.  This 
informational  equivalence  ensured  that  comparisons  between  the  RAPTOR  and  baseline 
interface  were  more  meaningful.  The  following  list  briefly  summarizes  baseline  interface 
functions,  tools,  and  displays: 

•  Situational  Awareness  Display  (see  Figure  22)  -  provides 

representations  for  terrain,  friendly  unit  icons,  and  enemy  icons  in  the 
same  manner  as  RAPTOR’S  Spatial  Synchronization  Display.  The 
primary  difference  between  these  two  displays  is  that  the  friendly  unit 
icons  presented  in  the  baseline  display  do  not  change  colors  corresponding 
to  their  categorical  status. 

•  FIPR  Reports  (see  Figure  26)  -  Contact  reports  and  SPOTREPs  provide 
users  with  enemy  specific  data  such  as  equipment  type  (e.g.,  tanks  and 
APCs),  location,  and  activity.  E-BDA  reports  provide  data  on  disabled 
enemy  resources  by  specific  type.  These  three  reports  provide 
alphanumeric  data  that  is  equivalent  to  the  enemy  data  represented  in 
RAPTOR’S  Enemy  Resource  Display.  Furthermore,  the  LOGSTAT 
reports  provide  detailed  alphanumeric  data  on  the  same  resources  that  are 
represented  in  RAPTOR’S  Friendly  Combat  Resource  Display.  The 
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•  Command  Directive  Button  (see  Figure  22)  -  clicking  on  the  command 
directive  button  activates  a  screen  with  radio  buttons  that  enables  users  to 
select  pre-planned,  alternate  COAs  (see  Figure  27).  This  function  is 
similar  to  the  COA  buttons  located  on  the  bottom  of  RAPTOR’S  screen. 
However,  unlike  RAPTOR,  baseline  interface  users  are  not  able  to 
preview  alternative  COAs.  Clicking  and  executing  a  COA  radio  button  in 
the  command  directives  menu  directs  subordinate  elements  to  conduct  the 
COA  selected. 

•  Applications  Button  (see  Figure  20)  -  clicking  on  the  applications  button 
activates  a  screen  with  radio  buttons  that  enable  users  to  display  enemy 
and  friendly  weapon  envelopes  (see  Figure  28).  This  function  is  similar  to 
the  control  buttons  located  on  the  top  of  RAPTOR’S  screen. 


AfPUCAHONS  SETTWCS 
FWENOU  WEAPON  ENVELOPE:  Q 
Ef€MYKEAPOf>  ENVELOPE:  0 

HOE  COMBAT  RESOURCES  ICONS:  0 

Kan;  coca 

Figure  27.  Additional  Baseline  Interface  Options  and  Tools. 

Conversely,  RAPTOR  provides  certain  forms  of  data  that  are  not  represented  by 
the  baseline  interface.  Examples  of  these  are  the  temporal  synchronization  display,  the 
alternative  COA  review  buttons,  and  the  force  ratio  display.  However,  these  data  are 
provided  to  participants  in  each  tactical  scenario’s  OPORD.  Specifically,  Annex  C  for 
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each  OPORD  provides  a  synchronization  matrix  (see  Appendices  A  and  B)  that  meshes 
subordinate  unit  tasks  with  higher-level  purposes  that  assists  users  with  anticipating  and 
coordinating  combat  activities.  Annex  C  for  each  OPORD  also  provides  alternative  COA 
concept  sketches  (see  Appendices  A  and  B)  that  enables  users  to  review  preplanned, 
alternative  CO  As.  Furthermore,  force  templates  depicting  the  enemy’s  most  likely  COA 
(MLCOA)  and  most  dangerous  COA  (MDCOA)  are  located  in  each  OPORD ’s  situation 
paragraph  (i.e.,  Paragraph  1).  Combining  these  enemy  templates  with  the  friendly  force 
composition  illustrated  by  Task/Organization  charts  located  at  the  beginning  of  each 
OPORD  enable  users  to  calculate  force  ratio  estimates  (see  Appendices  A  and  B). 

D.  TACTICAL  SIMULATION  MODELS 

Three  tactical  scenarios  were  developed  for  this  study.  The  attack  scenario  is  a 
simulated  conventional  high  intensity  conflict  in  desert  terrain  and  was  based  on  training 
exercises  conducted  at  the  U.S.  Army’s  National  Training  Center  at  Fort  Irwin, 
California.  The  raid  scenario  is  a  simulated  counter-insurgency  (COIN)  low  intensity 
conflict  in  urban  terrain  and  was  based  on  combat  operations  routinely  conducted  by 
U.S.  Army  forces  in  Iraq  and  Afghanistan.  The  defense  scenario  is  a  conventional  high 
intensity  conflict  in  desert  terrain  and  was  developed  for  training  purposes  only.  The 
intent  for  incorporating  the  defense  scenario  was  to  familiarize  participants  with  the 
functions,  tools,  and  displays  provided  by  either  the  RAPTOR  or  baseline  interface  while 
they  conducted  C  for  a  practice  trial.  The  following  sections  provide  a  detailed 
description  for  the  two  scenarios  (attack  and  raid)  utilized  during  experimental  events. 
The  friendly  and  enemy  activity  data,  resource  values  and  parameters,  algorithms,  and 
representations  described  in  this  section  were  used  to  populate  DMSC  Ovals  1-3. 

1.  Friendly  Situation 

The  friendly  forces  represented  in  each  scenario  consisted  of  a  Battalion-sized 
element  configured  as  a  Task  Force  (TF).  The  TF  maintained  the  same  task-organization 
for  each  scenario,  and  consisted  of  four  company  teams  (TM)  and  two  specialty  platoons 
(see  Figure  28).  TM  A  was  mixed  with  eight  Abrams  tanks  and  four  Bradley  Fighting 
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Vehicles  (BFV).  TM  B  was  pure  with  12  Abrams  tanks.  TM  C  (mixed)  contained  eight 
BFVs  and  four  Abrams  tanks.  TM  D  (pure)  contained  12  BFVs.  There  were  three 
platoons  (Platoons  1,  2,  and  3)  in  each  TM,  and  each  platoon  consisted  of  four  tactical 
vehicles  (either  all  tanks  or  BFVs).  The  two  specialty  platoons  remained  under  TF 
control  and  included  a  Mortar  platoon  consisting  of  four  120mm  self-propelled  mortars, 
and  a  Scout  platoon  equipped  with  four  High  Mobility  Multipurpose  Wheeled  Vehicles 
(HMMWV). 


TWA 

TM  B 

TM  C 

1 /A/Tank  PLT 

1/B/Tank  PLT 

1/C/MechlN  PLT 

2/A/Tank  PLT 

2/B/Tank  PLT 

2/C/Mech  IN  PLT 

3/C/Meci  IN  PLT 

3/S/Tank  PLT 

3/A/"ank  PLT 

TWO 

TF  Mortars 

TF  Scouts 

l/C'/Meci  IN  FLT 

A/Sac  (2  x  120 Tim) 

A/Wieeled  Sec 

2/D/Meci  IN  FLT 
3/0/Meci  IN  FLT 

B/S?c  (2  *  120tim) 

B/Wi§§led  Sac 

Figure  28.  Friendly  Force  Task-Organization  Chart. 

The  TF  mission  during  the  attack  scenario  was  to  destroy  enemy  forces  located 
within  the  TF’s  battlespace.  Tactical  tasks  included  locating  the  forward  edge  of  the 
enemy’s  obstacle  belt,  establishing  an  attack  by  fire  position  with  TM  D,  establishing 
multiple  breach  lanes  through  the  enemy  obstacle  belt,  conducting  a  forward  passage  of 
lines,  and  completing  the  destruction  of  enemy  forces  within  a  specified  objective. 

Three  possible  friendly  COAs  were  planned  for  the  attack  scenario.  The 
implementation  of  a  given  COA  was  dependent  upon  specific  critical  event  criteria  being 
met  at  a  DP.  The  first  TM  to  establish  a  breach  lane  through  the  enemy  obstacle  belt  was 
the  DP  criteria  for  the  attack  scenario  (see  Figure  29).  COA  A  (the  default  COA)  was 
predicated  on  TM  B  establishing  the  first  breach  lane,  and  planned  for  TM  D  to  assault 
the  objective  from  the  center.  COA  B  (the  preferred  COA)  was  predicated  on  TM  C 
establishing  the  first  breach  lane,  and  planned  for  TM  D  to  assault  the  objective  from  the 
south.  COA  C  was  predicated  on  TM  A  establishing  the  first  breach  lane,  and  planned 


for  TM  D  to  assault  the  objective  from  the  north. 
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—  —  —  —  — *  CO  A  A  (Default) 

—  —  —  —  COAO  (Pieremtd) 

—  - — *•  COA  C 

i4rop 

|  ^  4B  |  Obstacle  Bell 


PL  JEFFERSON  PL  RAMPAGE  PL  DAMAGE  PL  PHOENIX 


The  TF  mission  during  the  raid  scenario  was  to  disrupt  enemy  insurgent 
operations  within  the  fictional  town  of  A1  Icia  Maria.  Tactical  tasks  included  conducting 
raids  against  multiple  specified  objectives,  destroying  insurgent  activity  centers,  clearing 
numerous  avenues  of  approach,  neutralizing  a  high  value  individual  (FIVI),  and 
exfiltrating  from  the  battlespace. 

Three  possible  friendly  CO  As  were  also  planned  for  the  raid  scenario.  The  DP 
criteria  for  implementing  a  given  COA  in  this  scenario  was  dependent  upon  the  HVI’s 
location  within  the  battlespace  (see  Figure  30).  COA  A  (the  default  COA)  was 
predicated  on  the  HVI  being  located  at  Objective  Dylan  and  required  TM  D  to  neutralize 
the  HVI  while  TM  B  completed  the  enemy  disruption  by  raiding  Objective  Bruce.  COA 
B  was  predicated  on  the  HVI  not  being  located  within  A1  Icia  Maria,  and  required  TM  C 
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to  complete  the  enemy  disruption  by  raiding  Objective  Bruce.  COA  C  (preferred  COA) 
was  predicated  on  the  HVI  being  located  in  the  vicinity  of  Objective  Bruce,  and  required 
TM  A  to  neutralize  the  HVI  at  Objective  Bruce. 

- -*  COA  A  (Default) 

COA  B 

—  —  —  —  -♦  COA  C  (Preferred) 


Figure  30.  Courses  of  Action  for  Raid  Scenario. 

Both  scenarios  portray  the  friendly  missions  as  ongoing  combat  operations.  Thus, 
the  TF  had  to  execute  their  assigned  tactical  tasks  with  reduced  combat  resources.  This 
constraint  required  participants  to  manage  resource  expenditures,  and  to  maintain 
awareness  of  combat  resource  statuses  down  to  the  CO  level  to  ensure  goals  were 
achieved.  Table  2  depicts  the  initial  and  projected  categorical  resource  statuses  at  the  CO 
and  PLT  levels,  and  the  initial  parameter  values  established  for  individual  combat 
resources  at  the  beginning  of  both  scenarios. 
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Table  2.  Combat  Resource  Chart  for  Tactical  Scenarios  (Friendly  Forces). 


Company  Level 
Resource  Status 


Platoon  Level  Resource 
Status 


TM  A 


TM8 


CATEGORICAL 

Initial 

Projected 

Individual  Resource  Parameters 
(Initial  Values  Only) 


As  can  be  seen  in  Table  2,  friendly  combat  resources  consisted  of  tanks,  BFVs, 
mortars,  HMMWVs,  ammunition,  and  fuel.  Four  of  these  parameters  (tanks,  BFVs, 
mortars,  and  HMMWVs)  were  computed  as  a  simple  percentage  of  the  full  complement. 
For  simplification  purposes,  individual  unit  commander  and  executive  officer  vehicles 
were  not  included  in  the  scenarios.  Ammunition  was  computed  as  the  number  of 
potential  armored  vehicle  and  prepared  defensive  position  kills  (120mm  tank  rounds, 
120mm  mortar  rounds,  anti-tank  missiles,  25mm  rounds,  and  40mm  grenades).  Fuel  was 
computed  as  the  range  in  kilometers,  and  consumption  rates  were  based  on  each 
individual  vehicle’s  fuel  economy.  Though  RAPTOR  is  also  designed  to  consider 
humans  as  a  separate  combat  resource,  personnel  were  included  with  the  vehicles,  and 
crewmembers  were  considered  expended  as  friendly  vehicles  were  destroyed. 

2.  Enemy  Situation 

Unlike  the  friendly  forces,  enemy  force  composition  and  capabilities  were  not  the 
same  for  the  two  scenarios.  This  was  due  to  the  different  operational  environments  and 
conditions  used  to  create  the  simulation  models  (conventional  high  intensity  conflict  in 
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open  desert  terrain  vs.  COIN  low  intensity  conflict  in  urban  terrain).  The  enemy  also 
faced  numerical  and  technological  disadvantages  on  both  battlefields.  Therefore,  enemy 
forces  employed  other  types  of  “low-tech”  weaponry  to  close  these  disadvantageous 
“gaps”  in  both  scenarios. 

The  enemy  force  represented  in  the  attack  scenario  consisted  of  a  CO(+)  sized 
element.  Their  composition  included  three  PLTs  of  infantry  fighting  vehicles  (BMP-2) 
reinforced  by  one  PLT  of  T-72  tanks.  Each  PLT  consisted  of  three  tactical  vehicles 
(either  all  tanks  or  BMP-2s).  The  enemy’s  command  vehicle  (BMP-2)  was  also  present 
on  the  battlefield.  Figure  3 1  illustrates  the  enemy  composition  for  the  attack  scenario. 


I  <+) 


v  3  x  T-72  (MDCOA1 

3  x  BMP-2  per  PLT  (ML  CO  A) 


Figure  3 1 .  Enemy  Composition  Diagram  (Attack  Scenario). 


The  enemy  mission  during  the  attack  scenario  was  to  deny  friendly  forces  the 
ability  to  attack  west.  The  enemy  conducted  a  defense  in  depth  from  dug-in  fighting 
positions  to  increase  their  survivability.  They  employed  their  reserve  T-72  tank  platoon 
on  the  battlefield  and  conducted  a  counter-attack  into  the  friendly  force’s  exposed 
northern  flank  (see  Figure  32).  The  enemy  also  established  a  large  complex  obstacle  belt 
consisting  of  antitank  mines  and  concertina  wire  to  reduce  friendly  force 
numerical  superiority. 
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Figure  32.  Enemy  Most  Dangerous  COA  (Attack  Scenario). 


The  enemy  force  represented  in  the  raid  scenario  also  consisted  of  a  CO(+)  sized 
element.  However,  unlike  the  attack  scenario,  the  enemy  in  the  raid  scenario  is  an 
unconventional  insurgent  force  operating  either  as  individuals,  or  in  small  teams 
consisting  of  3-4  personnel.  Lone  enemy  elements  employed  suicide  car  bomb  attacks 
using  vehicle  borne  improvised  explosive  devices  (VBIED).  Enemy  teams  conducted 
anti-armor  ambushes  and  limited  indirect  fire  attacks.  Figure  33  illustrates  the  enemy 
composition  for  the  raid  scenario. 

I  (♦) 

♦ 


■  ■ •  •  aa  . 

♦ 

4  x  Ambush  PLT  3  x  VGIED  1  x  Mortar  TM  1  x  Sniper 

Figure  33.  Enemy  Composition  Diagram  (Raid  Scenario). 
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The  enemy  mission  during  the  raid  scenario  was  to  enable  the  HVI  to  exfiltrate 
from  the  battlespace  by  delaying  friendly  force  penetration  into  insurgent  support  zones. 
The  enemy  conducted  multiple  anti-armor  ambushes  from  prepared  fighting  positions 
located  within  several  structures  to  increase  their  survivability.  The  enemy  attacked 
friendly  forces  with  VBIEDs  and  indirect  mortar  fires  (see  Figure  34).  The  enemy  also 
employed  anti-tank  mines  and  numerous  improvised  explosive  devices  (IED)  (i.e., 
roadside  bombs)  to  reduce  friendly  force  numerical  and  technological  superiority. 


Figure  34.  Enemy  Most  Dangerous  COA  (Raid  Scenario). 


Enemy  combat  resources  represented  in  RAPTOR’S  enemy  combat  resource 
display  consisted  of  tanks  and  BMP-2s  for  the  attack  scenario.  VBIEDs  and  anti-armor 
ambush  teams  were  represented  in  the  display  for  the  raid  scenario.  For  simplicity 
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purposes,  VBIEDs  were  given  T-72  tank  force  equivalence,  and  the  anti-armor  ambush 
teams  were  given  BMP-2  force  equivalence.  IEDs  and  antitank  mines  possessed 
equivalent  lethality.  However,  enemy  ammunition  and  fuel  values  were  not  included. 

3.  Other  Simulation  Data 

Detailed  friendly  and  enemy  capability  data  (e.g.,  weapon  ranges,  weapon  re-load 
times,  vehicular  speeds,  etc.)  were  added  to  API  intelligence  algorithms  to  better  replicate 
actions  and  conditions  typically  encountered  in  the  physical  environment.  Vehicular  fuel 
capacity,  fuel  economy,  and  ammunition  combat  loads  were  also  added  to  more 
accurately  calculate  combat  resource  parameter  values  and  expenditure  rates  (see 
Table  3). 


Table  3.  Capability  Data  Matrix  (Friendly  and  Enemy). 


Probability  kill  (Pk)  data  was  also  added  to  the  tactical  simulation  models.  As  can 

be  expected,  friendly  forces  were  more  survivable  and  lethal  than  enemy  forces  in  the 

tactical  scenarios.  However,  the  Pk  values  established  for  enemy  forces  were  only 
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slightly  less  than  the  Pk  values  established  for  friendly  forces.  This  helped  to  better 
balance  friendly  versus  enemy  survivability  and  lethality.  Pk  algorithms  consisted  of  a 
simple  calculation  for  each  individual  weapon  system’s  probability  hit  *  each  tactical 
vehicle’s  probability  damage  expectancy  (Pk  =  Ph  *  Pde)-  Furthermore,  a  random  number 
generator  for  kinetic  exchanges  between  friendly  and  enemy  vehicles  was  also 
incorporated  to  ensure  one  force  did  not  possess  an  overwhelming  survivability-lethality 
advantage  over  the  other  force.  Table  4  illustrates  the  Pk  values  established  for  the 
tactical  simulation  models. 


Table  4.  Probability  Kill  Matrix  (Friendly  &  Enemy). 
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E.  PROCEDURES 

Participants  completed  three  sessions  (training,  trial  event  1 ,  and  trial  event  2)  on 
successive  days.  As  stated  earlier,  participants  were  randomly  assigned  to  four  groups 
(RAPTOR  Group  1,  RAPTOR  Group  2,  Baseline  Group  1,  and  Baseline  Group  2). 
Groups  were  blocked  on  one  type  of  interface.  The  Tactical  Scenario-Interface 
combinations  were  counter-balanced  to  minimize  order  effects.  The  following  sections 
illustrate  how  each  session  was  conducted. 
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a. 


Selection  &  Training 


A  convenience  sample  of  volunteers  was  recruited  from  various 
departments  within  NPS.  Each  volunteer  was  provided  with  a  brief  description  of  the 
study  and  asked  to  complete  at  demographic  survey  (see  Appendix  C).  Those  volunteers 
who  answered  “yes”  to  colorblindness  (question  6)  were  not  included  in  the  study.  Upon 
completing  the  demographic  survey  and  the  consent  form,  participants  were  randomly 
assigned  to  an  interface  group,  and  scheduled  for  a  training  session. 

Group  training  sessions  (i.e.,  RAPTOR  Interface  training  and  Baseline 
Interface  training)  were  conducted  prior  to  the  experimental  trails.  Participants  only 
attended  training  for  the  type  of  interface  to  which  they  were  assigned.  Each  training 
session  lasted  approximately  one  hour,  and  all  participants  received  an  oral  tutorial  of 
their  respective  interface,  and  a  written  and  oral  description  of  the  simulations.  Oral 
tutorials  and  descriptions  were  scripted  to  ensure  consistency  of  instruction  between  the 
different  groups.  The  tutorials  familiarized  participants  on  the  menus,  displays,  tools,  and 
functions  offered  by  their  assigned  interface  to  minimize  learning  effects  during  the  trials. 
Researchers  conducted  the  tutorial  using  a  pre-recorded  PowerPoint  presentation 
displayed  by  a  55-inch  flat  panel  liquid  crystal  display. 

Following  the  tutorial,  participants  conducted  a  practice  tactical  scenario 
(i.e.,  defense  scenario)  using  their  assigned  interface  to  further  minimize  learning  effects. 
Participants  were  given  sufficient  time  to  become  comfortable  with  manipulating  the 
various  tools  provided  by  their  specific  interface. 

Once  participants  completed  the  practice  scenario,  researchers 
administered  a  brief  knowledge  test  related  to  specific  display  options,  tools,  graphs, 
charts,  etc.  to  ensure  each  participant  retained  the  knowledge  required  for  proficient  use 
of  their  assigned  interface  type  (see  Appendix  D  &  E).  Proficiency  was  defined  as  a 
perfect  score  (100%)  with  every  question  answered  correctly.  Those  participants  who 
failed  to  score  a  100%  were  provided  with  an  opportunity  to  receive  further  training  and 
to  retake  the  knowledge  test  until  they  were  proficient. 

Upon  successful  completion  of  the  knowledge  test,  participants  were 
scheduled  for  their  first  trial.  Participants  were  also  provided  with  an  advanced  hard 
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copy  of  the  OPORD  pertaining  to  the  initial  tactical  scenario  they  would  encounter  to 
ensure  they  had  adequate  time  to  familiarize  themselves  with  the  scenario  and  to 
formulate  any  questions  prior  to  conducting  the  trial. 

b.  Experimental  Sessions 

Participants  conducted  trials  on  an  individual  basis.  Researchers  provided 
the  participant  with  a  tabbed  copy  of  the  tactical  OPORD,  the  TRACE  tool,  report 
formats,  other  associated  materials,  and  general  instructions  for  the  trial  once  they  arrived 
at  the  lab.  Participants  were  given  the  opportunity  to  review  the  OPORD  and  receive 
clarification  on  mission  specifics  and  anything  else  related  to  the  conduct  of  the  scenario. 
Participants  then  provided  researchers  with  a  mission  back  brief  to  ensure  they 
understood  the  mission,  commander’s  intent,  infonnation  requirements,  and  key  tasks  to 
be  executed  during  the  simulation.  Upon  successful  completion  of  the  back  brief,  the 
participants  conducted  the  25-minute  tactical  scenario. 

Participants  were  queried  for  specific  levels  of  SA  during  three  separate 
periods  within  the  tactical  scenarios.  Each  query  was  initiated  by  a  pre-recorded  audio 
prompt  requesting  a  SITREP.  The  simulation  paused  at  the  beginning  of  the  prompt,  and 
participants  were  alloted  five  minutes  to  collect  infonnation  for  as  many  TRACE  line 
item  entries  as  possible.  The  simulation  remained  paused  until  participants  reported 
answers  for  the  line  item  entries  they  were  able  to  complete.  Participants  were  able  to 
access  data  from  the  various  interface  displays  and  menus  during  the  pauses.  The 
purpose  for  these  pauses  was  to  enable  participants  to  concentrate  their  efforts  on 
collecting,  integrating,  and  reporting  queried  information  instead  of  being  forced  to 
divide  their  attention  between  preparing  TRACE  responses  while  also  trying  to  monitor 
ongoing  activities  occuring  on  their  screen.  Upon  completion  of  the  SITREP, 
participants  rated  their  perceived  level  of  accuracy  between  0%  to  100%  for  their 
TRACE  line  item  answers,  then  resumed  the  simulation  by  selecting  the  “done”  button 
located  on  the  bottom  of  their  screen. 

Participants  were  instructed  to  answer  specific  CCIRs  as  events  transpired 
in  the  simulations.  Each  scenario  contained  four  discrete  CCIR  activities  that  occurred  at 
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various  times,  phases,  and  locations  throughout  the  tactical  scenarios.  Participants 
annotated  and  reported  the  activities  once  they  perceived  the  cues.  Unlike  the  TRACE 
queries,  the  simulations  were  not  paused  for  CCIR  reporting. 

Participants  were  also  instructed  to  report  critical  event  criteria  linked  to 
DPs  in  the  tactical  scenarios.  Each  scenario  contained  one  DP  tied  to  three  pre-planned 
CO  As.  Unlike  the  CCIR  protocol  described  earlier,  participants  were  not  required  to 
report  each  individual  critical  event  as  they  occurred,  but  annotated  and  reported  the 
decision  made  at  a  DP.  In  other  words,  decisions  at  a  DP  reflected  participant 
comprehension  for  how  the  collective  occurrence  of  critical  events  impacted  the  friendly 
and  enemy  situation,  as  well  as  their  projection  for  how  they  expected  future  events  to 
unfold.  Participants  simultaneously  (or  near  simultaneously)  executed  the  COA  by 
selecting  the  corresponding  COA  button  on  their  screen.  Like  the  CCIR  reporting, 
simulations  were  not  paused  during  critical  event  decision  reporting  and  COA  selection. 
Critical  event  matrixes  pertaining  to  each  tactical  scenario  were  provided  to  the 
participants  as  an  annex  in  the  tactical  OPORDs  (see  Appendices  A  and  B). 

As  discussed  earlier,  participants  were  asked  to  enter  their  perceived  level 
of  cognitive  workload  on  the  subjective  workload  scale  when  prompted.  The  scale  was 
presented  every  five  minutes  during  each  25-minute  scenario,  and  remained  active  for 
30  seconds.  If  a  participant  failed  to  enter  their  perceived  workload  within  the  30 
seconds  allotted,  the  scale  disappeared  from  the  screen  and  the  participant  was  assigned  a 
“very  high”  (i.e.,  7)  score  for  that  estimation  period.  The  rationale  behind  this  scoring 
technique  was  based  on  an  assumption  that  the  participant  was  too  busy  to  momentarily 
divert  their  attention  toward  the  estimation  scale.  The  subjective  workload  estimation 
scale  was  presented  to  participants  even  when  the  simulation  was  paused  for  the  TRACE 
queries  to  maintain  consistency. 

The  OPORDs  used  for  the  experimental  events  were  packaged  in  a  three 
ring  binder  and  tabbed  for  quick  access  to  key  information.  Participants  were  allowed  to 
reference  an  OPORD  at  any  time  during  the  scenarios,  but  were  instructed  to  close  the 
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binder  after  they  accessed  the  information  of  interest.  Researchers  annotated  the  number 
of  times  participants  referenced  an  OPORD  during  a  scenario  as  another  simple 
workload  measure. 


Participants  were  scheduled  for  the  follow-on  trial  immediately  upon 
completing  the  initial  event.  The  procedures  for  the  subsequent  trials  were  the  same  as 
those  previously  described. 

Immediately  following  the  second  trial,  participants  assigned  to  the 
RAPTOR  groups  were  asked  to  complete  a  brief  feedback  survey  (see  Appendix  F).  The 
purpose  of  this  survey  was  to  elicit  participant  concerns  and  perceptions  about  the 
different  displays,  options,  and  tools  presented  by  the  RAPTOR  interface.  Participant 
feedback  was  used  to  compile  helpful  recommendations  aimed  at  improving  RAPTOR’S 
overall  design  and  to  develop  a  “way  ahead”  for  future  C  interface  studies. 
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IV.  RESULTS 


The  results  consist  of  three  parts:  (1)  statistical  analysis  for  TRACE 
measurements;  (2)  statistical  analysis  for  critical  infonnation  and  event  measurements; 
and  (3)  statistical  analysis  for  workload  measurements.  Summary  and  descriptive 
statistics  are  provided  in  parts  a,  b,  and  c  for  both  the  RAPTOR  and  Baseline  interface 
user  groups.  Inferential  statistics  are  used  in  all  parts  to  analyze  differences  between 
levels  of  situation  awareness,  decision  making,  and  workload  with  respect  to  query 
accuracy,  latency  times,  C-SWAT  inputs,  and  total  requests  for  infonnation  between  the 
RAPTOR  and  Baseline  interface  user  groups. 

A.  STATISTICAL  ANALYSIS  FOR  TRACE 

1.  TRACE  Latency 

Simulations  were  paused  three  times  during  each  scenario  for  TRACE 
information  collection  and  reporting.  Latency  was  measured  in  seconds  by  calculating 
when  the  simulation  paused  to  when  the  participant  reported  answers  for  completed  line 
item  and  sub-line  item  entries.  A  t-test  was  perfonned  to  compare  means  between  the 
two  scenarios;  there  was  no  evidence  that  a  learning  effect  had  occurred  as  participants 
advanced  from  one  trial  to  the  next  ( t  (30)  =  1.06,  p  =  .29).  Combined  mean  TRACE 
times  (i.e.,  attack  +  raid)  were  calculated  for  both  RAPTOR  and  Baseline  groups.  The 
RAPTOR  group  responded  more  quickly  with  an  overall  mean  latency  time  of  198.06 
seconds  (SD  =  35.77),  as  compared  to  an  average  latency  of  362.79  seconds  (SD  =  32.71) 
for  the  Baseline  group  (see  Table  5). 
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Table  5.  Descriptive  Statistics  for  TRACE  Latency. 


Baseline 

RAPTOR 

Mean 

362.79 

198.06 

Standard  Error 

8.18 

8.94 

Median 

363.67 

197.00 

Standard  Deviation 

32.71 

35.77 

Sample  Variance 

1069.81 

1279.25 

Kurtosis 

-0.19 

-0.15 

Skewness 

-0.35 

-0.04 

Range 

121.00 

132.00 

Minimum 

294.33 

137.00 

Maximum 

415.33 

269.00 

Consistent  with  the  research  design,  a  mixed  factor  ANOVA  was  performed  to 
test  for  differences  within  each  group  and  between  the  two  interfaces.  Test  results  found 
that  TRACE  latency  for  the  RAPTOR  group  was  significantly  less  than  the  Baseline 
group  {F(l,  14)  =  146.48,  p  <  .0001)  (see  Table  6). 


Table  6.  ANOVA  Results  for  Between  Interface  TRACE  Latency  Effects. 


Source 

Type  III  Sum  o' 

Squares 

df 

Mean  Square 

F 

Sig 

Intercept 

2516461  040 

1 

2516461  040 

1697  943 

000 

Interface  Design 

217086  136 

1 

217086  136 

146  476 

000 

Error 

20748  000 

14 

1482064 

Conversely,  results  also  indicated  that  no  significant  differences  existed  within 
each  group  (p  =  .16),  and  did  not  yield  evidence  of  a  significant  interface*scenario 
interaction  (p  =  .71)  (See  Figure  35). 
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Figure  35.  Estimated  Marginal  Means  for  TRACE  Latency. 


2.  TRACE  Accuracy 

TRACE  scores  were  calculated  as  correct  or  incorrect  for  each  line  item  and  sub¬ 
line  item  entry.  Correct  line  item  and  sub-line  item  entries  were  summed  to  detennine 
overall  TRACE  scores.  Participants  could  score  a  maximum  of  22  points  per  TRACE 
query  if  they  answered  all  line  item  and  sub-line  item  entries  correctly.  Another  t-test 
was  perfonned  to  compare  means  between  the  two  scenarios.  Results  from  this  test  also 
did  not  indicate  a  learning  effect  had  occurred  as  participants  advanced  from  one  trial  to 
the  next  ( t  (29)  =  -0.60,  p  =  .56).  The  RAPTOR  group  was  more  accurate  than  the 
Baseline  group  with  an  overall  mean  TRACE  score  of  21.54  (SD  =  0.53).  In  contrast,  the 
Baseline  group  had  an  overall  mean  TRACE  score  of  11.87  (SD  =  2.79).  The  small 
standard  deviations  coupled  with  the  medians  and  modes  being  relatively  close  to  the 
means  suggest  a  small  amount  of  variance  amongst  the  TRACE  scores  (see  Table  7). 


Table  7.  Descriptive  Statistics  for  TRACE  Scores 


Baseline 

RAPTOR 

Mean 

11.87 

21.54 

Standard  Error 

0.70 

0.13 

Median 

11.5 

21.67 

Mode 

11.33 

22 

Standard  Deviation 

2.79 

0.53 

Sample  Variance 

7.78 

0.28 

Kurtosis 

-0.71 

-0.97 

Skewness 

0.37 

-0.77 

Range 

8.67 

1.33 

Minimum 

8 

20.67 

Maximum 

16.67 

22 
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Results  from  a  mixed  factor  ANOVA  found  that  TRACE  scores  for  the  RAPTOR 
group  were  significantly  higher  than  Baseline  group  scores  (F(l,  14)  =  130.14,  p  < 
.0001)  (see  Table  8).  No  significant  differences  were  found  within  each  group  (p  =  .08), 
and  the  interface*scenario  interaction  was  also  not  significant  (p  =  .08)  (see  Figure  36). 


Table  8.  ANOVA  Results  for  Between  Interface  TRACE  Score  Effects. 


Source 

Type  III  Sum  of 

Squares 

df 

Mean  Square 

F 

Sig 

Intercept 

8933  500 

1 

8933  500 

1554  772 

OOO 

Interface  Type 

747  781 

1 

747  781 

130  143 

OOO 

Error 

80  442 

14 

5746 

HA/’T  OW 

10  •  A 
Artxb 

Typf 

Figure  36.  Estimated  Marginal  Means  for  TRACE  Scores. 

B.  STATISTICAL  ANALYSIS  FOR  CRITICAL  INFORMATION  AND 
EVENTS 

1.  CCIR  Latency 

Four  cues  pertaining  to  critical  information  requirements  were  presented  during 
each  scenario.  Latency  was  measured  in  seconds  from  when  the  critical  information  was 
available  to  when  the  participant  reported  the  CCIR  answer.  A  t-test  was  used  to 
compare  means  between  the  two  scenarios;  there  was  no  evidence  of  a  learning  effect  as 
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participants  advanced  from  one  trial  to  the  next  ( t  (29)  =  1.28,  p  =  .21).  Combined  mean 
CCIR  times  (i.e.,  attack  +  raid)  were  calculated  for  both  RAPTOR  and  Baseline  groups. 
The  RAPTOR  group  had  the  lowest  overall  mean  latency  time  of  38.14  seconds  (SD  = 
46.58)  as  compared  to  an  average  latency  of  107.56  seconds  (SD  =  66.54)  for  the 
Baseline  group  (see  Table  9). 


Table  9.  Descriptive  Statistics  for  CCIR  Latency. 


Baseline 

RAPTOR 

Mean 

107.56 

38.14 

Standard  Error 

16.64 

11.64 

Median 

98.13 

30 

Standard  Deviation 

66.54 

46.58 

Sample  Variance 

4427.80 

2169.49 

Kurtosis 

-0.21 

13.87 

Skewness 

0.84 

3.61 

Range 

208.75 

203.75 

Minimum 

28.75 

4 

Maximum 

237.5 

207.75 

Researchers  also  performed  a  mixed  factor  ANOVA  to  test  for  differences  within 
each  group  and  between  the  two  interfaces.  Test  results  found  that  CCIR  latency  for  the 
RAPTOR  group  was  significantly  less  than  the  Baseline  group  (F(l,  14)  =  14.47,  p  = 
.002)  (see  Table  10). 


Table  10.  ANOVA  Results  for  Between  Interface  CCIR  Latency  Effects. 


Source 

Type  III  Sum  of 

Squares 

df 

Mean  Square 

F 

Sig 

Intercept 

169835  205 

1 

169835  205 

63  724 

000 

Interface  Design 

38555  174 

1 

38555  174 

14  466 

002 

Error 

37312  402 

14 

2005  172 

Results  from  the  ANOVA  also  indicated  that  no  significant  differences  existed 
within  each  group  (p  =  .64),  and  did  not  yield  a  significant  interface*scenario  interaction 
(p  =  .32)  (See  Figure  37). 
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Figure  37.  Estimated  Marginal  Means  for  CCIR  Latency. 


2.  CCIR  Accuracy 

CCIR  cues  were  treated  as  discrete  activities.  Answers  for  each  of  the  four 
CCIRs  presented  per  scenario  were  scored  as  either  correct  or  incorrect.  Thus, 
participants  could  score  either  1  point  per  activity  if  they  answered  a  CCIR  query 
correctly  or  0  points  per  activity  if  they  answered  a  CCIR  query  incorrectly.  The  four 
scores  were  summed  for  each  scenario  with  total  possible  outcomes  ranging  between  0-4 
points.  The  RAPTOR  group  had  the  highest  overall  mean  CCIR  score  of  3.81  (SD  = 
0.14)  as  compared  to  an  average  score  of  2.13  (SD  =  0.81)  for  the  Baseline  group.  Once 
again,  the  small  standard  deviations  coupled  with  the  medians  and  modes  being  relatively 
close  to  the  means  suggest  a  small  amount  of  variance  amongst  the  TRACE  scores  (see 
Table  11). 


Table  11.  Descriptive  Statistics  for  CCIR  Scores. 


Baseline 

RAPTOR 

Mean 

2.13 

3-81 

Standard  Error 

0.20 

0-14 

Median 

2 

4 

Mode 

2 

4 

Standard  Deviation 

0.81 

0.54 

Sample  Variance 

0.65 

0-30 

Kurtosis 

0.75 

9-09 

Skewness 

0.63 

-3-03 

Range 

3 

2 

Minimum 

1 

2 

Maximum 

4 

4 

The  data  violated  the  normality  assumption,  thus  a  nonparametric  pennutation  test 

was  performed  to  compare  outcomes  between  RAPTOR  and  Baseline  groups.  The  test 
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consisted  of  a  simulation  using  S-Plus  statistical  software.  During  the  simulation, 
observed  samples  (i.e.,  CCIR  scores  per  participant)  were  randomly  distributed  between 
two  groups  of  size  eight  1,000  times  to  determine  how  often  the  re-sampled  statistic  of 
interest  was  as  extreme  as  the  observed  value  of  -27.  Results  of  the  simulation 
demonstrated  a  difference  as  extreme  as  (+/-)  27  only  one  time  out  of  1,000,  thus 
enabling  the  researchers  to  infer  a  statistically  significant  difference  existed  between  the 
two  interfaces.  Figure  38  illustrates  the  difference  of  observed  CCIR  scores  between  the 
RAPTOR  and  Baseline  interfaces. 


CCIR  Se«r«ft 


0  J  ?  «  *  . 


RAFIOR  CCIR  Sceres 


Figure  38.  Graph  of  RAPTOR  &  Baseline  CCIR  Scores 


3.  Critical  Event  Latency 

One  decision  point,  the  critical  event,  was  located  within  each  scenario.  Critical 
event  latency  was  measured  in  seconds  from  when  all  critical  event  criteria  had  been  met 
at  a  decision  point  to  when  the  participant  selected  a  specific  COA.  Combined  mean 
CCIR  times  (i.e.,  attack  +  raid)  were  calculated  for  both  RAPTOR  and  Baseline  groups. 
The  RAPTOR  group  had  the  lowest  overall  mean  critical  event  latency  time  of  81.19 
seconds  (SD  =  63.01)  as  compared  to  an  average  latency  of  94.27  seconds  (SD  =  32.63) 
for  the  Baseline  group.  However,  the  large  standard  deviation  coupled  with  the  median 
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and  mode  being  relatively  far  apart  from  the  mean  suggests  a  large  amount  of  variance 
between  times  within  the  RAPTOR  group  (see  Table  12). 


Table  12.  Descriptive  Statistics  for  Critical  Event  Times. 


Baseline 

RAPTOR 

Mean 

34.27 

81.19 

Standard  Error 

8.42 

15.75 

Median 

95 

58.5 

Mode 

119 

16 

Standard  Deviation 

32.63 

63.01 

Sample  Variance 

1064.50 

3970.30 

Kurtosis 

-1.04 

0.13 

Skewness 

0.04 

0.92 

Range 

110 

217 

Minimum 

43 

4 

Maximum 

153 

221 

Furthermore,  results  from  a  mixed  factor  ANOVA  found  no  statistical  difference 
in  critical  event  times  between  the  two  interfaces  (F(l,  14)  =  .127,  p  =  .73). 
Interestingly,  test  results  did  enable  researchers  to  discover  that  a  significant 
interface* scenario  interaction  existed  ( F(1 ,  14)  =  7.868,  p  =  .02)  (see  Figure  39). 


Figure  39.  Estimated  Marginal  Means  for  Critical  Event  Fatency. 
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c. 


STATISTICAL  ANALYSIS  FOR  WORKLOAD 


1.  Continuous  Subjective  Workload  Assessment  Technique  (C-SWAT) 

Perceived  levels  of  cognitive  workload  data  were  produced  from  participant 
entries  on  a  7-point  Likert  scale  (i.e.,  1  =  very  low  workload,  7  =  very  high  workload) 
every  5  minutes  during  each  scenario.  The  RAPTOR  group  had  the  lowest  overall  mean 
C-SWAT  entry  of  2.53  (SD  =  1.01)  as  compared  to  an  average  entry  of  4.9  (SD  =  1.00) 
for  the  Baseline  group  (see  Table  13). 


Table  13.  Descriptive  Statistics  for  C-SWAT  Entries. 


Baseline 

Ft  A  PTOFt 

Mean 

4.9 

2.53 

Standard  Error 

0.25 

0.25 

Med  I  a  n 

4. as 

2.2 

Mode 

5.9 

1.4 

Standard  Deviation 

1.00 

1.01 

Sample  Variance 

0.09 

1.02 

Kurtosis 

-0.86 

-0.34 

Skewness 

0.02 

0.74 

Range 

3-4 

3.3 

Minimum 

3.1 

1.4 

Maxim  Li m 

6-S 

4.7 

Since  C-SWAT  scores  were  ordinal,  a  Mann-Whitney  test  was  performed  to 
examine  differences  in  C-SWAT  means  between  the  RAPTOR  and  Baseline  groups. 
Test  results  found  that  C-SWAT  scores  for  the  RAPTOR  group  was  significantly  less 
than  the  Baseline  group  during  both  the  attack  (z  =  -3.00,  p  =  .003)  and  raid  (z  =  -2.90,  p 
=  .004)  scenarios  (see  Table  14). 

Table  14.  Mann-Whitney  Results  for  Mean  C-SWAT  Entry  Differences. 


Attack 

Raid 

Mann-Whitnev  U 

3.500 

4.500 

Z 

-2.998 

-2.899 

Asymp.  Siq.  <2-tailed> 

.003 

.004 

txact  Siq  |2'(  1  -tailed  Siq.)| 

out* 

.002* 
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Figure  40  further  illustrates  C-SWAT  entry  differences  by  scenario  between  the 
two  interfaces. 


hiian  M»it* 


Figure  40.  Average  C-SWAT  Scores  for  Baseline  and  RAPTOR  Groups. 


2.  Requests  for  Information  (RFI) 

A  tally  was  made  for  each  time  a  participant  referred  to  an  OPORD  during  a 
scenario.  These  tallies  resulted  in  a  total  RFI  count  at  the  end  of  each  scenario. 
Combined  mean  RFI  counts  (i.e.,  attack  +  raid)  were  calculated  for  both  RAPTOR  and 
Baseline  groups.  The  RAPTOR  group  had  the  lowest  overall  mean  RFIs  of  0.44  (SD  = 
0.51)  as  compared  to  an  average  RFI  tally  of  6.25  (SD  =  1.65)  for  the  Baseline  group. 
Once  again,  the  small  standard  deviations  coupled  with  the  medians  and  modes  being 
relatively  close  to  the  means  suggest  a  small  amount  of  variance  amongst  the  RFIs  (see 
Table  15). 


Table  15.  Descriptive  Statistics  for  RFI  Counts. 


flose//ne 

Ft  A  F*-TOFt 

Mean 

6.25 

0.4375 

Standard  Error 

0.41 

0.13 

Median 

6 

O 

M  ode 

6 

O 

Standard  Deviation 

1.65 

0.51 

Sample  Variance 

2.73 

0.26 

Kurtosls 

-0.11 

-2.22 

Skewness 

-0.05 

0.28 

Range 

6 

1 

Minimum 

3 

o 

Maximum 

9 

1 
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A  mixed  factor  ANOVA  was  used  to  test  for  differences  within  each  group  and 
between  the  two  interfaces.  Results  found  that  the  RAPTOR  group  had  significantly 
fewer  RFIs  than  the  Baseline  group  (F(l,  14)  =  194.67,  p  <  .0001)  (see  Table  16). 


Table  16.  ANOVA  Results  for  Between  Interface  RFI  Count  Effects. 


SUUILW 

Type  III  Sum  of 

Sgutti  es 

or 

Mean  S  qutiie 

F 

Siy. 

intercept 

35/  /81 

1 

35/  781 

25/695 

UOU 

Interface  Design 

270  281 

1 

270  281 

194  672 

OOO 

hrror 

iy  438 

14 

1  388 

Results  from  the  ANOVA  also  indicated  that  a  significant  difference  did  exist 
within  groups  (F(l,  14)  =  13.51,  p  <  .002),  but  did  not  yield  a  significant 
interface* scenario  interaction  (p  =  .10)  (See  Figure  41). 


Figure  4 1 .  Estimated  Marginal  Means  for  RFI  Counts. 

D.  STATISTICAL  ANALYSIS  FOR  PERCEIVED  VS.  ACTUAL  ACCURACY 

Participants  were  asked  to  indicate  their  confidence  they  had  in  the  accuracy  of 
their  TRACE  line  item  answers  during  each  simulation  pause.  This  was  called 
“Perceived  Accuracy”  and  ranged  from  0%  to  100%.  Actual  accuracy  data  were 
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produced  from  the  total  number  of  line  item  entries  answered  correctly.  The  RAPTOR 
group  reported  and  achieved  the  highest  overall  mean  accuracy  percentage  of  0.94  (SD  = 
0.04)  and  0.98  (SD  =  0.02)  respectively  (see  Table  17). 


Table  17.  Descriptive  Statistics  for  Perceived  and  Actual  Accuracy. 


Baseline 

RAPTCR 

Perceived 

Actual 

Perceived 

Actual 

Mean 

0.86 

0.54 

C.94 

0.93 

Slandard  Error 

0.02 

0.04 

C.01 

0.01 

Median 

0.87 

0.52 

0.94 

0.93 

Standard  Deviation 

0.07 

0.10 

0.04 

0.02 

Sample  Variance 

0.00 

0.01 

0.00 

0.01 

Kurtosis 

3.03 

-1.21 

-1.28 

2.95 

Skewness 

-1.52 

0.00 

0.25 

-1.59 

Range 

0.22 

0.29 

0.12 

0.07 

Minimum 

0.72 

0.38 

C.88 

0.93 

Maximum 

0.93 

0.67 

1.00 

1.07 

Another  mixed  factor  ANOVA  was  used  to  test  for  differences  within  each  group 
and  between  the  two  interfaces.  Results  found  that  perceived  and  actual  TRACE 
accuracy  percentages  for  the  RAPTOR  group  was  significantly  higher  than  the  Baseline 
group  (F(l,  14)  =  89.76,  p  <  .0001)  (see  Table  18). 


Table  18.  ANOVA  Results  for  Between  Interface  Confidence  Effects. 


Source 

Type  III  Sum  of 

Squares 

df 

Mean  Square 

F 

Sifl. 

Intercept 

21  978 

1 

21  978 

3755  281 

OOC 

Interface  Deskjn 

.525 

1 

525 

89  756 

ooc 

Error 

082 

14 

006 

Furthermore,  results  from  the  ANOVA  indicated  that  a  significant  difference 
existed  within  groups  (F(l,  14)  =  54.06,  p  <  .0001),  and  that  a  significant 
group* accuracy  interaction  also  existed  (F(l,  14)  =  90.98,  p  <  .0001)  (see  Figure  42). 
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Figure  42. 
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V.  DISCUSSION 


This  chapter  discusses  the  implications  of  the  results  presented  in 
Chapter  IV  and  discusses  why  such  significant  findings  consistently  emerged.  Key 
observations  gathered  from  other  researchers’  work  discussed  throughout  Chapter  II  are 
included  to  emphasize  military  relevance  of  the  findings. 

A.  IMPLICATIONS  OF  ANALYSIS 

1.  Situation  Awareness  (Hypothesis  1) 

•  Hai:  The  RAPTOR  interface  leads  to  better  levels  of  SA  than  the 
U.S.  Army’s  FBCB2  interface. 

The  TRACE  tool  was  developed  to  provide  researchers  with  a  non-obtrusive 
method  for  collecting  participant  SA  data.  TRACE  latency  and  accuracy  results  show 
that  RAPTOR  users  were  able  to  answer  TRACE  queries  significantly  faster  and  more 
accurately  than  Baseline  interface  users,  which  supports  the  first  hypothesis  (see  Figure 
43). 


Figure  43.  Combined  TRACE  Results  (Latency  and  Accuracy). 


Participant  responses  to  TRACE  queries  were  used  to  populate  the  Dynamic 
Model  of  Situated  Cognition  (DMSC)  Ovals  4  to  6.  As  stated  in  Chapter  II,  answers  to 
TRACE  Lines  1-3  provided  information  on  Level  1  SA  for  populating  Oval  4 
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(Perception).  Answers  to  Lines  4-6  provided  information  on  Level  2  SA  for  populating 
Oval  5  (Comprehension),  while  answers  to  Line  7  provided  information  on  Level  3  SA 
for  populating  Oval  6  (Projection).  RAPTOR  enabled  participants  to  correctly  answer  an 
average  of  21.54  out  of  22  sub-line  item  queries  in  an  average  of  198  seconds.  In 
contrast,  Baseline  interface  participants  correctly  answered  an  average  of  only  1 1.87  sub- 
line  item  queries  in  an  average  of  363  seconds  (see  Tables  5  and  7).  Baseline  interface 
users  were  only  able  to  perceive  and  gain  limited  comprehension  for  how  deviations 
could  potentially  endanger  current  task  achievement,  while  RAPTOR  users  required 
significantly  less  time  to  successfully  forecast  how  future  events  would  potentially 
impact  objectives  and  end  states.  These  findings  directly  support  research  conducted  by 
Shattuck  and  Miller  (2006)  who  found  that  the  design  of  an  interface  can  affect  SA  by 
representing  the  environment  more  or  less  accurately. 

RAPTOR  was  designed  to  support  direct  perception  by  taking  advantage  of 
powerful  human  perceptual  resources  by  presenting  friendly,  enemy,  and  environmental 
data  in  a  meaningful,  coherent,  and  structured  manner  (Rasmussen,  1992).  Conversely, 
the  Baseline  interface  did  not  appear  to  support  direct  perception.  The  Baseline  interface 
presented  complex  data  primarily  through  alphanumeric  reports.  While  RAPTOR’S 
design  seemed  to  decrease  the  amount  of  cognitive  resources  required  to  acquire  and 
integrate  the  data  presented,  the  Baseline  interface’s  design  forced  users  to  apply 
extensive  cognitive  resources  to  reason  about  situations.  This  inference  is  further 
supported  by  the  ad-hoc  tables  and  matrices  created  on  notepaper  by  Baseline  users 
during  the  experimental  trials  as  a  strategy  to  cope  with  task  demands  (see  Appendix  G). 
The  successful  results  from  this  study  strategy  support  Talcott  et  al.’s  (2007) 
recommendation  that  the  incorporation  of  an  intact  perception-action  loop  should  be 
considered  as  a  higher-order  goal  in  interface  designs. 

Enhanced  levels  of  SA,  as  demonstrated  by  RAPTOR  users,  contribute  to  flexible 
and  agile  forces  that  are  capable  of  acting  faster  than  the  enemy  (Bushey  &  Forsyth, 
2006).  Improved  TRACE  speed  and  accuracy  help  operators  close  the  “information  gap” 
which,  according  Endsley  and  Garland  (2000),  is  an  important  criterion  for  assessing  the 
benefit  of  any  tactical  C  system  interface  design. 
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2. 


Decision  Making  (Hypothesis  2) 


•  Hai:  The  RAPTOR  interface  supports  better  decision-making  processes 
than  the  U.S.  Army’s  FBCB2  interface. 

a.  Critical  Information  Inferences 

Commanders  Critical  Information  Report  (CCIR)  latency  and  accuracy 


results  show  that  RAPTOR  users  were  able  to  answer  critical  information  queries 
significantly  faster  and  more  accurately  than  Baseline  interface  users,  which  supports  the 
second  hypothesis  (see  Figure  44). 


Figure  44.  Combined  CCIR  Results  (Latency  and  Accuracy). 


Participant  responses  to  CCIR  queries  are  represented  by  feedback  loops 
in  the  DMSC  (see  Figure  7).  RAPTOR  enabled  participants  to  correctly  answer  an 
average  of  3.8  out  of  4  CCIR  queries  in  an  average  of  38  seconds.  In  contrast,  Baseline 
interface  participants  correctly  answered  an  average  of  only  2.1  sub-line  item  queries  in 
an  average  of  108  seconds  (see  Tables  9  and  11).  The  speed  at  which  RAPTOR  users 
were  able  to  perceive  and  process  critical  information  provided  them  with  additional  time 
to  confirm  or  deny  expectations.  Again,  these  findings  directly  support  research 
conducted  by  Shattuck  and  Miller  (2006)  who  found  that  the  design  of  an  interface  can 
affect  decision  making  by  representing  the  environment  more  or  less  accurately. 

Reasons  for  these  results  can  be  attributed  to  the  design  principle  of  direct 
perception  as  previously  described.  However,  RAPTOR  was  also  designed  to  support 
direct  manipulation,  thereby  maintaining  intact  perception-action  loops  and  allowing 
operators  to  act  directly  on  objects  of  interest  in  the  interface  (Talcott  et  al.,  2007; 
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Bennett  et  al.,  2008).  In  contrast,  the  Baseline  interface  design  supports  primarily 
indirect  manipulation  which  results  in  inefficient  action  sequences  (Talcott  et  ah,  2007). 
This  problem  is  especially  evident  when  the  map  display  is  covered  by  the  large  pop-up 
windows  (see  Figures  23  and  24).  Consequently,  the  findings  generated  by  CCIR 
measurements  support  Shattuck  et  al.’s  (2000)  conclusions.  That  is,  interfaces  that  make 
salient  the  most  important  data  enables  commanders  to  focus  on  significant  portions  of 
the  battlefield  and  enables  them  to  reason  about  situations  in  a  more  sophisticated 
manner. 

With  respect  to  military  relevance,  the  speed  and  accuracy  with  which 
RAPTOR  can  enable  commanders  to  identify  and  comprehend  critical  infonnation 
requirements  facilitates  timely  decision-making  processes  that  potentially  affect 
successful  mission  accomplishment  (Department  of  the  Anny,  2004).  The  CCIR  findings 
suggest  that  RAPTOR  has  the  potential  to  enable  commanders  to  operate  within  enemy 
decision  making  cycles,  which  will  lead  to  agile  forces  capable  of  acting  faster  than  the 
enemy. 


b.  Critical  Event  Inferences 

Comparisons  of  critical  event  latency  between  the  two  interfaces  were  not 
significant.  To  provide  participants  with  an  opportunity  to  make  decisions  during  the 
experimental  trials,  the  researchers  elected  to  draft  tactical  scenarios  containing  three 
COAs.  One  of  these  COAs  had  to  be  selected  by  the  time  friendly  forces  met  all  critical 
event  criteria  at  a  decision  point.  Only  one  decision  point  was  located  within  each 
scenario.  Unlike  the  TRACE  or  CCIR  queries,  no  signals  or  mechanisms  were 
incorporated  into  any  of  the  interface  displays  to  assist  participants  with  making  the 
decision.  COA  selections  were  based  solely  on  participants  understanding  the  criteria 
listed  in  scenario  decision  support  matrices  and  their  temporal  recognition  of  critical 
events  as  they  transpired  in  the  battlespace.  Consequently,  no  participants  chose  an 
incorrect  COA. 

This  outcome  may  seem  odd  given  the  significant  differences  found  in  the 
levels  of  SA  and  critical  infonnation  comprehension  between  the  two  interfaces. 
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However,  this  result  does  support  Adams  et  al.’s  (1995)  finding  that  it  is  possible  for 
commanders  with  minimal  SA  to  implement  timely  and  accurate  decisions  because  their 

experience  and  training  may  be  sufficient  to  offset  degraded  views  of  the  situation  as  long 

2 

as  they  understand  task  demands.  All  participants  were  very  experienced  C 
practitioners,  and  all  had  been  queried  by  researchers  prior  to  conducting  scenarios  to 
ensure  they  fully  understood  the  mission,  commander’s  intent,  information  requirements, 
and  key  tasks  to  be  executed  during  the  simulations.  Thus,  individual  participant 
experience,  training,  and  scenario  understanding,  coupled  with  the  scenarios  containing 
only  one  decision  point  may  have  led  to  a  ceiling  effect. 

Interestingly,  results  indicated  a  significant  interface* scenario  interaction. 
Essentially,  Baseline  interface  users  selected  COAs  during  the  raid  scenario  faster  than 
RAPTOR  users.  On  the  surface,  this  outcome  seems  inconsistent  with  perfonnance 
patterns  demonstrated  by  Baseline  users  throughout  all  other  tasks.  However,  when 
taking  into  consideration  that  all  Baseline  users  had  combat  deployments  to  either  Iraq  or 
Afghanistan  (from  which  the  raid  scenario  was  modeled),  and  all  had  previous  FBCB2 
experience,  it  is  plausible  to  infer  that  the  participants  decided  upon  one  COA  prior  to  the 
decision  point  since  the  scenario  was  sufficiently  representative  of  situations  in  which 
they  had  recent  exposure  and  experience.  This  argument  has  implications  for  Klein’s 
(1993)  RPD  model,  which  states  that  while  under  time  pressure  commanders  rely  on  past 
experiences  to  select  their  COA. 

3.  Workload  (Hypothesis  3) 

•  Ha^:  The  RAPTOR  interface  requires  less  cognitive  workload  than  the 
U.S.  Army’s  FBCB2  interface 

C-SWAT  scores  and  RFI  reference  results  show  that  cognitive  workload  was 
significantly  less  for  RAPTOR  users  than  Baseline  interface  users,  which  supports  the 
third  hypothesis  (see  Figure  45). 
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Figure  45.  Combined  Workload  Results  (C-SWAT  and  RFI). 


Participant  C-SWAT  scores  were  used  to  measure  self-reported  cognitive 
workload.  Participants  using  RAPTOR  reported  an  average  workload  of  2.5  out  of  7 
(i.e.,  1  =  very  low,  7  =  very  high).  In  contrast,  Baseline  interface  participants  reported  an 
average  workload  of  5  (see  Table  13).  Researchers  also  annotated  the  number  of  times 
participants  referenced  an  OPORD  during  the  scenarios  as  an  additional  workload 
measure.  The  argument  for  using  this  measurement  was  that  referring  to  external  forms 
of  information  forced  participants  to  divide  their  attention  between  monitoring  interface 
displays  and  accessing  additional  information,  which  could  contribute  to  increased 
workload.  Participants  using  RAPTOR  sought  external  information  an  average  of  0.5 
times  per  scenario.  Conversely,  Baseline  interface  participants  sought  external 
information  an  average  of  6.3  times  per  scenario  (see  Table  15). 

The  design  of  RAPTOR  was  driven  by  the  explicit  consideration  of  the  C2  work 
domain  (Bennett,  et  ah,  2008).  Abstraction  and  aggregation  hierarchies  and  SRK 
taxonomy  principles  (Vincente  &  Rasmussen,  1992)  permitted  RAPTOR  users  to  capture 
critical  data  pertaining  to  tactical  scenario  goals,  purposes,  and  constraints,  because 
RAPTOR  represented  that  information  in  the  form  of  higher  and  intermediate  levels  of 
visual  salience  directly  on  the  display  (Talcott  et  ah,  2007).  Information  presented  in  this 
manner  enabled  participants  to  focus  on  critical  information  (e.g.,  combat  power, 
resources,  time,  task  synchronization,  and  force  ratios)  without  having  to  switch  between 
multiple  sets  of  displays.  In  contrast,  the  Baseline  interface  presented  data  primarily  via 
alphanumeric  reports,  which  forced  participants  to  access  numerous  menus,  tabs,  and 
individual  unit  reports  to  gather  the  data.  Unlike  RAPTOR,  whose  individual  displays 
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provide  users  with  continuously  updated  aggregate  and  categorical  values  of  key  data, 
Baseline  interface  users  had  to  calculate  numerous  parameter  values  (either  manually  or 
mentally)  after  the  necessary  data  were  obtained  (see  Appendix  G  for  examples  of  the  ad- 
hoc  tables  and  matrices  created  by  Baseline  users  during  the  experimental  trials).  These 
results  support  Talcott  et  al.’s  (2007)  research  which  found  that  displays  having 
aggregation  and  abstraction  principles  incorporated  into  their  designs  support  human 
perception  of  infonnation  in  time  and  space,  ultimately  reducing  operator  workload. 

Researchers  also  found  evidence  of  significant  differences  within  groups  for  RFI 
results.  Analysis  revealed  that  RAPTOR  and  Baseline  participants  made  more  references 
to  external  infonnation  during  the  raid  scenario  than  the  attack  scenario.  This  result  is 
not  surprising  when  considering  that  the  goals  and  objectives  identified  for  non- 
conventional  tactical  operations  are  often  more  ambiguous  and  confusing  than  those 
identified  for  conventional  operations. 

With  respect  to  military  relevance,  reduced  cognitive  workload  enables 
commanders  to  acquire  and  maintain  higher  levels  of  SA  (Adams  et  ah,  1995;  Wickens, 
2008).  Reduced  cognitive  workload  also  frees  the  commander  to  spend  his  time  and 
resources  on  higher-level  cognitive  processes,  which  may  reduce  uncertainty  and  lead  to 
better  decisions. 

4.  Perceived  vs.  Actual  Accuracy 

Just  prior  to  the  beginning  of  data  collection,  researchers  decided  to  ask 
participants  about  their  perceived  accuracy  (i.e.,  0%  =  very  low,  100%  =  very  high)  in 
their  TRACE  responses.  This  decision  was  based  on  the  researchers’  intuition  and  was 
intended  to  provide  insights  into  future  research  in  this  area.  Although  the  researchers 
had  no  infonned  hypotheses  about  what  the  data  would  yield,  the  results  proved  very 
interesting. 

The  results  show  that  RAPTOR  users  were  significantly  more  accurate  in  their 
TRACE  answers  than  Baseline  interface  users.  However,  an  even  more  interesting 
discovery  was  the  significant  difference  found  within  the  Baseline  group.  Essentially, 
Baseline  users  reported  being  considerably  more  accurate  in  their  TRACE  answers  than 
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they  actually  were.  Perhaps  most  interesting  was  the  significant  group* accuracy 
interaction.  Baseline  users  perceived  themselves  to  be  very  accurate,  but  actually 
achieved  low  accuracy.  Conversely,  RAPTOR  users  perceived  themselves  to  be  less 
accurate,  but  actually  achieved  high  accuracy. 

Because  researchers  decided  to  collect  perceived  accuracy  data  late  in  the  study, 
no  research  pertaining  to  confidence  or  trust  in  automation  was  conducted  prior  to  the 
data  being  collected  and  analyzed.  Therefore,  we  were  unable  to  provide  concrete 
explanations  for  why  these  results  may  have  occurred. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  Conclusions  and  Recommendations  chapter  addresses  four  key  areas 
pertaining  to  RAPTOR.  These  comments  are  applicable  to  C  technologies  in  general. 
The  key  areas  are:  (1)  Study  Conclusions;  (2)  Future  Research;  (3)  Recommendations; 
and  (4)  Final  Comments.  The  Study  Conclusions  section  will  discuss  important 
information  ascertained  from  the  study,  while  the  Future  Research  section  provides  a 
“way-ahead”  for  RAPTOR’S  continued  development.  The  Recommendations  section 
focuses  on  modifications  that  should  be  considered  for  future  versions  of  RAPTOR  to 
help  make  the  interface  even  more  effective  for  warfighter  use.  And,  the  Final 
Comments  section  provides  the  researchers’  final  thoughts  about  this  study. 

A.  STUDY  CONCLUSIONS 

Results  from  this  study  indicate  that  the  RAPTOR  interface  was  more  effective 
than  the  Baseline  interface  in  all  areas  examined.  Six  out  of  seven  statistical  comparisons 
between  the  interfaces  were  significant,  suggesting  that  performance  with  the  RAPTOR 
interface  was  better  than  perfonnance  with  the  Baseline  interface.  More  importantly,  the 
pattern  of  results  found  in  this  study,  coupled  with  the  results  found  during  previous 
studies,  clearly  indicate  that  the  theoretical  principles  used  to  create  RAPTOR  provide  a 
very  effective  interface  design  strategy  for  assisting  military  practitioners  in  coping  with 
the  complexities  and  uncertainties  inherent  in  C  .  Though  no  interface  will  result  in 
complete  understanding  or  perfect  SA,  RAPTOR  has  demonstrated  its  ability  to 
effectively  support  warfighter  cognitive  processing  while  reducing  workload,  and  may 
also  prove  to  be  a  significant  enabler  in  assisting  the  U.S.  Anny  with  maintaining  a 
tactical  edge  over  threat  forces. 

B.  FUTURE  RESEARCH 

This  study  was  bounded  within  C  activities  normally  conducted  during  the 
execution  phase  of  a  tactical  operation.  In  reality,  effective  C2  begins  during  the  planning 
and  preparation  phases,  where  specific  goals  are  defined  and  key  tasks  are  detennined, 
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and  continue  through  the  reconsolidation  phase  after  identified  tactical  goals  and 
objectives  have  been  achieved.  Accordingly,  RAPTOR’S  design  concept  encompasses  a 
holistic  approach  toward  assisting  commanders  with  C  throughout  all  phases  of  tactical 
operations.  Thus,  the  following  discussion  on  future  research  will  focus  on  three  areas  of 
study:  (1)  additional  research  on  operator  trust  and  confidence  when  using  RAPTOR  to 
assist  with  C  ;  (2)  RAPTOR’S  application  to  the  planning  and  preparation  phases  of 
tactical  operations;  and  (3)  RAPTOR’S  application  as  an  assessment  tool. 

1.  Additional  Research  on  Trust  and  Confidence 

The  preliminary  perceived  and  actual  accuracy  results  discussed  in  Chapters  IV 
and  V  warrant  additional  research  to  provide  plausible  explanations  for  why  RAPTOR 
users  were  less  confident,  yet  more  accurate  in  their  TRACE  answers,  while  Baseline 
users  were  more  confident,  yet  considerably  less  accurate  in  their  answers. 

As  stated  previously,  prior  research  pertaining  to  user  confidence  and/or  trust  in 
automation  was  not  conducted,  thus  researchers  refrained  from  speculating  about  why 
these  results  may  have  occurred.  However,  the  results  do  raise  several  interesting 
questions.  Perhaps  the  Baseline  users  were  overconfident  given  that  they  all  had  previous 
FBCB2  experience  in  tactical  environments.  Perhaps  RAPTOR  users  mistrusted  the 
RAPTOR  interface  given  the  novelty  of,  and  their  inexperience  with,  the  technology. 
Additional  research  in  this  area  may  provide  even  more  conclusive  evidence  on 
RAPTOR’S  ability  to  enable  warfighters  to  cope  with  complex  and  dynamic  situations. 

2.  Application  into  Planning  and  Preparation  Phases 

This  study  focused  on  RAPTOR’S  ability  to  enhance  warfighter  perfonnance 
during  the  execution  phases  of  tactical  operations.  However,  the  interface  is  designed  to 
assist  with  all  aspects  of  C  ;  additional  research  is  needed  to  detennine  the  extent  to 
which  RAPTOR  enhances  the  ability  of  commanders  and  their  staffs  to  plan  and  prepare 
for  tactical  operations. 

Planning  is  an  arduous  and  time-consuming  endeavor  that  requires  activities  such 
as  integration,  coordination,  and  synchronization  of  friendly  forces  and  battlefield 
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operating  systems.  During  this  phase,  countless  hours  are  dedicated  to  collecting  and 
calculating  (either  manually  or  mentally)  detailed  estimates  to  determine  the  effects  that 
numerous  interrelated  factors  (e.g.,  friendly  capabilities,  enemy  forces,  terrain,  weather, 
time,  etc.)  will  have  on  tactical  operations.  Accordingly,  RAPTOR  is  designed  to 
compute  many  of  the  same  types  of  data  normally  calculated  by  battle  staff  personnel. 
Also,  RAPTOR’S  various  displays  (e.g.,  friendly  combat  resource  display,  enemy  combat 
resource  display,  force  ratio  display,  and  temporal  synchronization  display)  represent  data 
in  tables,  charts,  and  graphs  that  are  similar  to  products  typically  generated  during 
mission  analysis  and  COA  development  processes. 

Battlefield  preparation  also  requires  continuous  estimate  refinement,  COA 
analysis  and  comparison,  and  approval  processes.  Similar  to  the  planning  phase,  many  of 
RAPTOR’S  displays  and  manipulable  tools  can  assist  battle  staffs  with  specific 
preparation  processes.  For  example,  RAPTOR’S  COA  buttons  enable  the  commander 
and  his  staff  to  preview  and  compare  differences  between  alternative  COAs  in  the  spatial 
synchronization  and  temporal  synchronization  modes.  The  graphical  replay  slider 
enables  the  commander  and  his  staff  to  preview  and  analyze  pre-planned  activities  in 
time  and  space.  RAPTOR  can  also  assist  with  refining  estimates  by  computing  fresh  data 
as  updates  are  received. 

Presumably,  the  speed  and  efficiency  afforded  by  RAPTOR  will  enable 
commanders  and  staff  personnel  to  spend  their  time  and  energy  on  higher-level  processes 
such  as  decision  making.  Thus,  studies  that  examine  RAPTOR-aided  planning  and 
preparation  processes  may  produce  conclusive  evidence  on  the  interface’s  ability  to 
enable  battle  staffs  to  receive,  process,  share,  disseminate,  and  display  reliable 
information  faster  and  more  effectively  than  current  C"  technologies. 

3.  Application  as  an  Assessment  Tool 

Commanders  must  assess  actions  taken  (or  not  taken)  during  every  phase  of  a 
tactical  operation  to  avoid  committing  similar  mistakes  during  future  operations,  and  to 
continuously  improve  overall  unit  performance.  The  after-action  review  (AAR)  is  a  type 
of  assessment  routinely  conducted  during  training  and  in  combat.  AARs  enable 
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commanders  to  identify  deficiencies,  sustain  proficiency,  and  focus  on  strengthening 
specific  task  perfonnance.  Effective  AARs  (i.e.,  those  that  uncover  and  capture  key 
lessons  learned)  explore  critical  events,  actions,  and  observations  by  time  sequence  to 
prevent  the  loss  of  valuable  information  and  to  promote  constructive  feedback 
(Department  of  the  Army,  1993). 

Consequently,  assessments  such  as  AARs  are  another  arena  in  which  RAPTOR 
may  be  suited  to  assist  commanders.  In  essence,  RAPTOR  records  graphical 
representations  of  events  as  they  transpire  in  time  and  space.  The  plan  review  mode  and 
graphical  replay  slider  provide  commanders  with  the  ability  to  “rewind”  through 
historical  events  and  locate  discrete  activities  of  interest  to  determine  exactly  which 
deviations  occurred  during  precise  points  in  time.  In  reality,  commanders  have  very 
limited  capabilities  to  capture  activities  during  combat  operations  in  Afghanistan  and  Iraq 
for  AAR  purposes.  Thus,  studies  that  examine  RAPTOR-aided  AAR  processes  may 
illustrate  the  interface’s  potential  to  enable  warfighters  to  ascertain  fine  details  of  crucial 
lessons  that  may  often  remain  unnoticed  during  current  battlefield  operations. 

C.  RECOMMENDATIONS 

The  following  sections  are  focused  on  researchers’  observations  and  participant- 
elicited  feedback  that  should  be  considered  for  future  versions  of  RAPTOR  to  help  make 
the  interface  more  intuitive  and  beneficial  for  warfighter  use.  Strategies  are  also 
described  for  generalizing  RAPTOR’S  capabilities  to  other  military  operations  beyond  the 
context  of  battalion-level  command  and  control. 

1.  Researcher  Observations 

a.  Display  Modifications 

Portions  of  RAPTOR’S  displays  must  become  more  robust  in  order  to 
represent  different  structures  and  capabilities  for  both  friendly  and  threat  forces.  The 
current  unit  control  tree  design  represents  friendly  units  as  an  armored  task  force 
configuration  typically  employed  by  Heavy  Brigade  Combat  Teams  (HBCT)  before  the 
Army’s  transformation  process  began  in  earnest  in  1999.  Since  then,  the  Army  has 
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fielded  Stryker  Brigade  Combat  Teams  (SBCT)  and  reconfigured  many  HBCTs  into 
Units  of  Action  (UA).  The  Army  also  currently  fields  Infantry  Brigade  Combat  Teams 
(IBCT),  Airborne  Brigade  Combat  Teams  (ABCT),  and  Armored  Cavalry  Regiments 
(ACR).  Each  are  uniquely  structured  and  equipped  with  considerably  different 
capabilities.  Furthermore,  combat  brigades  will  often  receive  additional  combat 
multipliers  such  as  attack  aviation,  field  artillery,  and  military  police  to  enable  mission 
accomplishment.  The  unit  control  tree  must  be  sufficiently  tailor-able  to  represent  the 
various  force  structures  and  combat  multipliers  employed  by  current  combat 
brigade  teams. 

Similarly,  the  current  enemy  combat  resource  display  represents  equipment 
primarily  associated  with  conventional  enemy  force  structures.  However,  rocket- 
propelled  grenades  and  improvised  explosive  devices  are  the  major  weapon  systems 
currently  used  by  insurgent  forces  in  Iraq  and  Afghanistan.  Thus,  future  RAPTOR 
versions  should  be  sufficiently  tailor-able  to  represent  a  wide  array  of  capabilities  that 
can  be  employed  by  conventional  and  non-conventional  threat  forces.  Also,  force 
equivalence  algorithms  must  be  accurately  reflect  friendly  and  enemy  force  structures  to 
ensure  force  ratio  values  are  properly  computed  and  presented  in  the  force  ratio  display. 

b.  Usability 

Currently,  information  represented  by  RAPTOR’S  displays  cannot  be 
altered  by  users.  Future  versions  must  include  intuitive  options  and  tools  that  enable 
users  to  quickly  and  efficiently  alter,  update  and  refine  infonnation  represented  in  the 
various  displays  as  situations,  conditions,  and  missions  change.  For  example,  specific 
tasks  and  timing  considerations  are  detennined  during  the  mission  planning  and 
preparation  phases.  Staff  personnel  must  be  able  to  populate  and  refine  synchronization 
points  and  activities  in  the  spatial  and  temporal  synchronization  displays  as  the  plan 
matures.  Staff  personnel  must  also  be  able  to  build  alternate  courses  of  action,  branches, 
and  sequels  into  the  same  displays  during  COA  development,  comparison,  and  analysis 
processes.  Furthermore,  the  unit  control  tree  and  enemy  resource  display  must  permit 
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staffs  to  accurately  represent  force  structures  as  friendly  combat  elements  and  multipliers 
are  attached  and/or  detached,  and  as  threat  capabilities  change. 

An  additional  capability  that  should  be  considered  for  incorporation  into 
future  versions  of  RAPTOR  is  a  tool  that  enables  staff  personnel  to  build  and  refine 
overlays  that  can  be  “laid”  on  top  of  maps  presented  in  the  spatial  synchronization 
display.  The  Army  routinely  uses  numerous  overlays  such  as  graphic  control  measures, 
tactical  mission  graphics,  and  modified  combined  obstacle  overlays  to  highlight  mission 
details  and  directives  that  require  special  emphasis  (Department  of  the  Army,  2004). 
RAPTOR  should  permit  users  to  build,  save,  access,  disseminate,  share,  and  display 
overlays  when  required.  Users  should  also  have  the  ability  to  layer  multiple  overlays  on 
top  of  the  map,  and  be  able  to  turn  specific  overlays  “on  or  off’  when  needed.  FBCB2 
enables  users  to  build  graphic  control  measures,  but  its  functionality  is  very  limited  in 
scope.  In  contrast  to  FBCB2,  the  robust  overlay  capability  described  is  currently 
supported  by  FalconView,  which  is  a  Windows-based  mapping  system  originally 
designed  for  U.S.  Air  Force  aviation  mission  planning.  However,  unlike  FalconView, 
RAPTOR  should  support  overlay  options  representative  of  symbols  and  colors  that  are  in 
accordance  with  U.S.  Anny  conventions. 

2.  Participant  Feedback 

Participants  who  used  the  RAPTOR  interface  during  experimental  events  were 
asked  to  complete  a  brief  feedback  survey  at  the  conclusion  of  their  final  trial.  The 
survey  (see  Appendix  F)  consisted  of  six  statements  about  the  different  displays,  options, 
and  tools  presented  by  the  RAPTOR  interface.  Participants  were  asked  to  score  how 
strongly  they  either  agreed  or  disagreed  with  each  statement  by  selecting  an  applicable 
number  on  a  5 -point  Likert  scale  (1  =  strongly  disagree  and  5  =  strongly  agree). 
Participants  were  also  encouraged  to  provide  comments  about  the  specific  displays, 
options,  and  tools  referenced  in  each  statement.  The  following  summarizes  participant 
comments  and  average  scores  provided  for  each  statement: 

•  The  individual  resource  bar  chart  color  codes  used  in  the  Friendly  Combat 
Resource  Display  enables  rapid  comprehension  of  unit  combat 
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effectiveness  (Average  Score  =  4.5).  Three  (out  of  16)  participants  stated 
that  certain  bar  colors  were  difficult  to  detennine  when  the  chart 
background  color  was  the  same  as  the  bar  color.  In  particular,  amber  bars 
tend  to  appear  gray  when  presented  on  a  chart  with  an  amber  background. 
Additionally,  one  participant  commented  that  the  resource  charts  would  be 
more  effective  if  all  pacing  items  (e.g.,  mortar  carriers,  self-propelled 
howitzers,  etc.)  in  a  unit  task  organization  were  added  as  additional 
combat  parameters. 

•  The  Force  Ratio  Display  facilitates  decision  making  by  enabling  users  to 
quickly  determine  which  force  (friendly  or  enemy)  has  a  superior 
advantage  (Average  score  =  4).  Five  participants  stated  that  the  force  ratio 
display  is  too  large,  and  the  value  of  the  data  represented  does  not  justify 
the  amount  of  space  dedicated  to  the  display.  Two  participants 
commented  that  the  force  ratio  is  great  for  planning  purposes,  but  during 
execution,  knowing  available  friendly  combat  power  is  more  important 
than  knowing  force  ratio  values. 

•  The  Unit  Control  Tree  enables  users  to  quickly  detennine  friendly 
resource  statuses  at  finer  or  courser  levels  of  detail  (Average  score  =  4.5). 
Two  participants  stated  that  the  unit  control  tree  is  a  very  useful  and 
intuitive  tool. 

•  The  COA  button  assists  with  decision  making  by  enabling  users  to  rapidly 
access  and  view  alternative  actions  friendly  forces  can  execute  if  required 
(Average  score  =  4.375).  Three  participants  stated  that  the  COA  review 
buttons  is  a  great  operational  tool,  but  that  changes  in  unit  activities 
between  each  COA  should  be  highlighted  in  the  temporal  synchronization 
display  to  better  enable  users  to  quickly  determine  major  differences.  It  is 
important  to  note  that  changes  between  COAs  are  highlighted  by  different 
color  synchronization  points  and  activity  lines  in  the  spatial 
synchronization  display.  One  participant  stated  that  additional  buttons 


111 


should  be  added  so  that  branch  plans  and  sequels  could  also  be  viewed  in 
the  spatial  and  temporal  synchronization  displays. 

•  Information  provided  in  the  Temporal  Synchronization  Display  enables 
users  to  anticipate  future  friendly  force  activities  by  time,  phase,  and  event 
(Average  score  =  4.5).  The  three  participants  made  the  same  statements  as 
above  about  changes  in  unit  activities  between  each  COA  should  be 
highlighted  for  easier  recognition. 

•  The  Enemy  Combat  Resource  Chart  reduces  uncertainty  by  enabling  users 
to  quickly  detennine  enemy  strength  and  combat  effectiveness  (Average 
score  =  4.25).  Three  participants  stated  that  the  enemy  resource  display  is 
a  very  intuitive  tool  that  greatly  assists  users  in  determining  enemy  battle 
damage  assessments  and  to  build  an  overall  mental  model  of  the  threat 
environment. 

The  below  statements  stem  from  the  final  part  of  the  survey  that  asked 
participants  to  provide  general  comments  about  RAPTOR’S  overall  usefulness: 

RAPTOR  is  much  easier  to  use  than  FBCB2.  The  displays  provide 
comprehensive  and  visual  data  representations  that  facilitate  quick  and 
accurate  decision  making  processes. 

RAPTOR  is  a  great  tool  that  has  the  potential  to  streamline  many  C~ 
processes.  Data  represented  by  the  different  displays  makes  decision 
making  much  easier,  and  the  color  codes  are  excellent  at  enabling  rapid 
battlefield  assessments. 

The  only  flaw  I  see  with  RAPTOR ’s  design  is  the  inability  to  communicate 
with  people  out  in  the  battlespace. 

The  statement  concerning  the  force  ratio  display  received  the  lowest  average 
score  and  also  generated  the  most  comments.  The  central  issues  were  (1)  force  ratios  are 
more  valuable  for  decision  making  during  the  planning  phase  and  less  valuable  for 
decision  making  during  execution  phase,  and  (2)  the  current  display  design  occupies  too 
much  space  that  could  be  used  to  display  other  types  of  information.  Perhaps  future 
versions  of  RAPTOR  should  incorporate  a  smaller  force  ratio  display.  The  additional 
space  could  be  used  for  branch  plan  and  sequel  review  buttons  as  suggested.  Human 
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interaction  concerns  could  also  be  addressed  by  adding  a  free  text  window  to  enable 
commanders  to  “chat”  with  subordinate  commanders.  If  a  smaller  force  ratio  display 
cannot  be  designed,  perhaps  users  could  be  provided  with  an  option  to  turn  the  display 
“on  or  off’  as  desired.  When  turned  off,  users  can  access  other  infonnation 
recommended  by  participants.  When  turned  on,  the  force  ratio  display  would  temporarily 
“mask”  the  additional  information  until  the  force  ratio  data  is  no 
longer  required. 

3.  Progression  Strategies 

The  theoretical  constructs  used  to  design  RAPTOR  may  enable  the  interface  to  be 
applied  to  other  military  operations  beyond  the  context  of  battalion  level  command  and 
control. 


a.  Application  to  Higher  Level  Commands 

This  study  was  bounded  within  C  activities  occurring  at  the  battalion 
level.  However,  battalions  normally  deploy  and  conduct  tactical  operations  as  a  part  of 
larger  brigade-size  organizations.  As  stated  earlier,  many  “legacy”  brigade  combat  teams 
have  been  restructured  into  UAs  to  better  fulfill  the  Army’s  expeditionary  needs.  As  a 
result,  UA  commanders  control  3  to  4  maneuver  battalions,  indirect  fire  units,  engineer 
assets,  and  a  wide  range  of  intelligence,  surveillance,  and  reconnaissance  capabilities 
(e.g.,  UAV  UGVs,  electronic  sensor  suites,  etc.)  in  order  to  orchestrate  multiple 
engagements  simultaneously.  Coordinating  and  synchronizing  various  formations  and 
platforms  designed  to  perfonn  distinctive,  yet  interdependent  roles  makes  C  at  brigade 
and  higher  levels  much  more  complex  and  dynamic  than  C  activities  conducted  at 
battalion  levels. 

Consequently,  the  Army  is  pursuing  a  Command  Post  of  the  Future 
(CPOF)  that  enables  commanders  and  staffs  to  bridge,  analyze,  and  correlate  disparate 
sources  of  data  originating  from  nodes  distributed  throughout  the  battlefield.  The  goal  of 
CPOF  technologies  is  to  aide  problem  solving  and  decision  making  by  packaging  and 
presenting  data  in  fonnats  that  support  human  thought  processes  (DARPA,  2009).  Goals 
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established  for  CPOF  could  potentially  be  achieved  by  implementing  RAPTOR’S 
theoretical  concepts  into  the  program.  RAPTOR’S  direct  perception,  manipulation, 
aggregation,  and  abstraction  hierarchy  design  principles  may  prove  invaluable  at  enabling 
decision-makers  to  achieve  desired  levels  of  information  processing,  integration,  and 
collaboration  throughout  ah  echelons  of  command. 

b.  Expansion  to  Other  Military  Services 

Scenarios  used  for  this  study  were  developed  specifically  for  U.S.  Army 
personnel.  However,  ah  military  services  must  cope  with  the  complexities  of  C  during 
combat  operations.  Since  each  service  strives  to  achieve  better  levels  of  SA  and 
enhanced  decision  making,  RAPTOR’S  design  principles  may  also  prove  useful  in 
assisting  U.S  Navy  carrier  group  commanders  during  continuous  operations  at  sea  or 
U.S.  Air  Force  commanders  during  extended  air  campaigns.  Combat  operations  executed 
by  the  U.S.  Marine  Corps  closely  parallel  those  executed  by  the  U.S.  Anny.  Presumably, 
RAPTOR’S  effectiveness  at  facilitating  C  activities  conducted  by  U.S.  Army  personnel 
may  also  prove  successful  for  U.S.  Marine  Corps  personnel.  Scenarios  representative  of 
distinct  U.S.  Marine  Corps  tactical  problems  (e.g.,  amphibious  assault  operations)  should 
be  developed  to  explore  RAPTOR’S  applicability  into  other  tactical  environments. 
Future  research  efforts  should  consider  RAPTOR’S  impact  on  the  planning  and  execution 
of  joint  military  operations. 

c.  Migration  to  Civilian  Occupations 

Effective  command  and  control  is  not  just  a  military  problem.  Many 
civilian  businesses  routinely  plan,  synchronize,  and  coordinate  complex  activities  to 
reduce  risks  and  ensure  best  business  practices.  In  particular,  the  transportation  industry 
(e.g.,  airlines,  railways,  trucking  companies,  etc.)  uses  sophisticated  technologies  to  plan 
efficient  operator  schedules,  movement  tables,  and  travel  routes  to  achieve  profitable 
transit  goals.  Furthermore,  finite  resource  expenditures  are  a  major  constraint  for 
transportation  planners  and  operators  considering  the  rising  price  of  gasoline  in  the 
current  economy.  Consequently,  transport  controllers  and  asset  operators  are  also 
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becoming  more  reliant  on  global  positioning  systems  to  track  delivery  progress  and  to 
avoid  potential  delays  caused  by  weather,  traffic,  and  other  types  of  unforeseen  events. 
However,  these  technologies  are  not  always  capable  of  producing  acceptable  solutions 
given  the  complex  and  dynamic  environments  in  which  transportation  occurs.  Operator 
intervention  is  often  required  to  solve  problems.  Although  the  operating  environments 
are  different,  the  defining  characteristics  of  the  objects  of  interest  are  similar  to  those  in 
military  C2.  Therefore,  it  is  reasonable  to  speculate  that  an  interface  founded  on 
ecological  and  CSE  design  principles  would  provide  a  very  effective  strategy  for  enabling 
the  transportation  industry  to  achieve  desired  goals  and  end  states. 

D.  FINAL  COMMENTS 

Infonnation  is  vital  for  success  during  war.  Those  who  are  faster  at  collecting, 
analyzing,  integrating,  and  understanding  relevant  infonnation  will  gain  a  superior 
advantage  over  any  adversary.  However,  the  quest  for  more  infonnation  can  also 
degrade  operational  effectiveness.  Research  into  technologies  designed  to  conectly 
support  human  cognition  has  great  potential  for  enhancing  warfighter  reasoning  and 
thought  processes,  while  at  the  same  time  reducing  operator  workload.  The  researchers 
are  confident  that  the  findings  in  this  study  will  lead  to  interface  designs  capable  of 
enhancing  military  practitioner  by  improving  SA,  resulting  in  better  decisions  during 

complex,  fluid,  and  dynamic  situations.  Additionally,  the  proposed  research  areas  may 

2 

also  provide  conclusive  evidence  of  RAPTOR’S  potential  to  facilitate  every  aspect  of  C 
throughout  all  levels  of  command  and  in  a  wide  range  of  operational  environments. 
Finally,  researchers  believe  results  of  this  study  will  assist  the  U.S.  Army  in  its  efforts  to 
develop  advanced  C  interfaces  that  account  for  human  capabilities  and  limitations. 
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APPENDIX  A.  ATTACK  SCENARIO  OPERATIONS  ORDER 


OPERATION  ORDER  08-43  (OPERATION  TYPHOON) 


References:  Map,  DMA,  1983,  Scale  1:50,000,  Series  V795,  Sheet  I 


Time  Zone  Used  Throughout  Order:  Local 


Task  Organization: 


TM  A  <SF) 

TM  R(SF) 

TMC(SF) 

1/A/Tank  PIT  (4  x  Ml) 

1 /8/Tank  PLT  (4  x  Ml) 

1 /C/Mach  IN  PLT  (4  x  BFV) 

2/A/Tank  PIT  (4  x  Ml) 

2/8/Tank  PLT  (4  x  Ml) 

2/C/Mech  IN  PLT  (4  x  BFV) 

3/C/Mech  IN  PLT  (4  x  BFV) 

3/8/Tank  PLT  (4  x  Ml) 

3/A/Tank  PIT  (4  x  Ml) 

TM  D  (ME) 

TF  Mortars 

TF  Scouts 

1/D/MeUi  IN  PLT  (4  a  BFV) 

A/3«ft.  (2  x  120mm) 

3«.(2a  HMMWV) 

2/D/Mech  IN  PLT  (4  x  BFV) 
3/D/Moeh  IN  PLT  (4  x  BFV) 

B/Sec  (2  x  120mm) 

B/Wheeted  Sec  (2  x  HMMWV) 

1.  SITUATION. 

a.  Battlefield  conditions. 

(1)  Weather.  No  Change. 

(2)  Light  Data.  No  Change 

(3)  Terrain.  Elevation  gradually  increases  to  the  west.  The  primarily  open  desert 
terrain  located  east  of  the  47  Easting  supports  large  formations  of  armored  vehicles 
traveling  at  high  rates  of  speed. 

(4)  Obstacles.  A  large  blocking  obstacle  consisting  of  anti-tank  mines  and  wire 
is  located  to  the  East  of  the  enemy  defensive  positions,  and  runs  from  North  to  South 
across  the  width  of  the  TF  zone.  Obstacle  belts  are  tied  into  the  ridges  located  in  the 
northern  and  southern  portions  of  the  TF  zone.  The  entire  length  of  the  blocking 
obstacle  is  covered  by  overlapping  direct  fires  from  enemy  forces  located  within  OBJ 
KILLER. 


b.  Enemy  Forces.  Immediately  opposing  our  TF  are  elements  of  the  269th 
Motorized  Rifle  Battalion  (MRBN)  estimated  at  100%  strength.  The  269th  is 
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defending  with  three  Motorized  Rifle  Companies  (MRC)  consisting  primarily  of 
BMP-2s,  and  is  also  believed  to  be  reinforced  by  one  CO  of  T-72  tanks.  Enemy 
defensive  positions  have  been  confirmed  at  OBJ  KILLER.  Though  no  specific 
combat  platforms  have  been  identified,  enemy  forces  projected  up  to  company 
strength  (10-13  vehicles)  have  been  templated  on  OBJ  KILLER. 

(1) .  MLCOA  in  TF  OUTLAW  Zone  [See  Appendix  1  (SITEMP)  to 
Annex  B  (Intelligence)].  Enemy  defends  in  depth  within  OBJ  KILLER  with  three  BMP 
PLTs  in  prepared  fighting  positions. 

(2) .  MDCOA  in  TF  OUTLAW  Zone.  Enemy  defends  in  depth  vie  OBJ 
KILLER  with  three  BMP  PLTs  and  one  T-72  Tank  PLT  in  prepared  defenses.  Enemy 
will  attempt  to  flank  the  TF  by  conducting  a  counter-attack  into  either  the  southern  or 
northern  flanks. 


(3).  Enemy  Composition  (Templated). 

c.  Friendly  Situation.  The  current  mission  is  part  of  ongoing  offensive 
operations.  Due  to  the  OPTEMPO,  logistical  are  experiencing  difficulties  with 
resupplying  forward  units.  Thus,  TF  OUTLAW  will  execute  this  mission  with  severely 
reduced  resources. 

2.  MISSION.  NLT  xxxxJUNxx,  TF  OUTLAW  attacks  west  toward  PL 
JEFFERSON  and  destroys  enemy  forces  located  vie  OBJ  KILLER  IOT  facilitate 
continued  offensive  operations  by  follow-on  forces. 

3.  EXECUTION. 

a.  OUTLAW  6  Intent: 

Purpose:  Complete  the  destruction  of  enemy  forces  in  TF  LUCKY 

zone. 

Key  Tasks: 

•  Establish  multiple  breach  lanes  through  enemy  obstacle  belt. 

•  Rapid  FPOL  of  TM  D  to  OBJ  DALLAS  [Decision  Point  1  (DP  1)]. 

•  Complete  the  destruction  of  enemy  forces  on  OBJ  KILLER  (TM  D). 

•  Establish  screen  along  PL  JEFFERSON. 
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Endstate: 

•  Friendly:  TF  OUTFAW  preparing  for  future  combat  operations  along  PF 
JEFFERSON. 

•  Enemy:  All  enemy  forces  destroyed  on  OBJ  KIEFER. 

b.  Concept  of  Operations  [See  Appendix  1  (Concept  Sketch)  to  Annex  C 
(Operations)].  The  decisive  point  of  this  operation  is  the  rapid  FPOF  of  TM  D  toward 
OBJ  DAFFAS  (DP  1)  IOT  complete  the  destruction  of  enemy  forces  located  within  OBJ 
KIEFER.  This  is  a  5  phase  operation:  (1)  PF  PHOENIX  to  PF  DAMAGE;  (2)  Breach; 
(3)  Assault  OBJ  KIEFER;  (4)  FPOF  to  OBJ  DAFFAS;  (5)  Screen.  Three  courses  of 
action  (COA)  have  been  planned  based  on  the  first  CO/TM  to  establish  a  breach  lane  and 
maneuver  west  of  PF  RAMPAGE.  The  following  conditions  drive  which  COA  will  be 
implemented  at  DP  1  [See  Appendix  3  (Decision  Support  Matrix)  to  Annex  C 
(Operations)]: 

•  COA  A  (initial  COA  to  be  executed)  -  TM  D  follows  TM  B  (center  zone  in  AXIS 

B)toward  OBJ  KIEFER. 

•  COA  B  -  TM  D  follows  TM  C  (south  zone  in  AXIS  C)  toward  OBJ  KIEFER. 

•  COA  C  -  TM  D  follows  TM  A  (north  zone  in  AXIS  A)  toward  OBJ  KIEFER. 

c.  Scheme  of  Maneuver  [See  Appendix  2  (Execution  Matrix)  to  Annex  C 
(Operations)]. 


(1)  PHASE  I  (PE  PHOENIX  to  PL  DAMAGE)  -  TF  Scouts  RP  PL 
PHOENIX  first,  and  conduct  moving  screen  along  the  northern  portion  of  the  TF  zone 
IOT  protect  the  TF  north  flank.  Sequentially,  once  TF  Scouts  reach  PL  DAMAGE,  TMs 
A,  B,  &  C  RP  PL  PHOENIX  and  attack  west  toward  PL  DAMAGE.  TM  A  attacks  along 
AXIS  A  in  the  north,  TM  B  attacks  along  AXIS  B  in  the  center,  and  TM  C  attacks  along 
AXIS  C  in  the  south.  Once  TMs  A,  B,  and  C  reach  PL  DAMAGE,  TM  D  departs  ATK 
POS  D  and  TF  Mortars  depart  CP  1  west  toward  PL  PHOENIX.  Phase  I  ends  once  TF 
Scouts  reach  PL  RAMPAGE,  TMs  A,  B,  and  C  are  arrayed  along  PL  DAMAGE,  and  TM 
D  with  TF  Mortars  executing  PL  PHOENIX  west  toward  PL  DAMAGE. 
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(2)  PHASE  II  (BREACH)  -  (DP  1  located  in  this  phase)  This  phase 
begins  once  TMs  A,  B,  and  C  execute  PL  DAMAGE  west  to  PL  RAMPAGE.  TF  Scouts 
establish  OP  1  on  the  key  terrain  located  vie  14RPV  462163  to  identify  forward  edge  of 
enemy  obstacles  and  enemy  defensive  positions  within  OBJ  KILLER.  TF  Mortars 
establish  a  mortar  firing  point  (MFP)  vie  CP  2  and  prepare  to  support  TF  breaching 
operations  with  indirect  fires.  TMs  A,  B,  and  C  locate  enemy  obstacles  vie  PL 
RAMPAGE  and  prepare  to  breach.  TF  Mortars  fire  TGT  GRP  CJ7  to  suppress  enemy 
forces  on  OBJ  KILLER.  Sequentially,  TM  D  establishes  ABF  D  and  destroys  enemy 
forces  located  vie  OBJ  KILLER  ISO  TF  breaching  operations.  TM  A  breaches  in  AXIS 
A,  TM  B  breaches  in  AXIS  B,  and  TM  C  breaches  in  AXIS  C.  TM  D  immediately 
collapses  ABF  D  once  the  first  CO/TM  establishes  a  breach  lane  and  maneuvers  west  of 
PL  RAMPAGE  (DP  1 ).  TM  D  follows  the  CO/TM  toward  OBJ  KILLER  (currently 
planned  as  COA  A  behind  TM  B).  TF  Scouts  remain  at  OP  1  to  observe  mortar  fires  ISO 
TF  breaching  operations.  Phase  II  ends  with  TM  D  assaulting  west  through  the  breach 
toward  OBJ  KILLER. 

•  Alternate  COAs  at  DP  1  -  In  the  event  TM  B  does  not  establish  the  first  breach 
lane,  TF  OUTLAW  prepares  to  execute  alternate  COAs  IAW  DP  1  criteria. 

o  COA  B  is  the  event  TM  C  establishes  the  initial  breach  lane  and 

maneuvers  west  of  PL  RAMPAGE.  During  COA  B,  TM  D  follows  TM 
C  south  toward  OBJ  KILLER. 


o  COA  C  is  the  event  TM  A  establishes  the  initial  breach  lane  and 

maneuvers  west  of  PL  RAMPAGE.  During  COA  C,  TM  D  follows  TM 
A  north  toward  OBJ  KILLER. 

(3)  PHASE  III  (ASSAULT  OBJ  KILLER)  -  This  phase  begins  once  all 
TMs  have  maneuvered  west  of  PL  RAMPAGE  toward  OBJ  KILLER.  TF  Mortars  fire 
TGT  DF0104  to  suppress  enemy  forces  located  vie  OBJ  DALLAS.  Simultaneously,  TM 
A  assaults  OBJ  ATLANTA  and  destroys  enemy  forces  located  in  the  northern  portion  of 
OBJ  KILLER.  TM  B  assaults  OBJ  BOSTON  and  destroys  enemy  forces  located  in  the 
forward  center  portion  of  OBJ  KILLER.  TM  C  assaults  OBJ  CHICAGO  and  destroys 
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enemy  forces  located  in  the  southern  portion  of  OBJ  KILLER.  TM  D  follows  and 
supports  TM  B  on  OBJ  BOSTON  (COA  A).  TF  Scouts  remain  at  OP  1  to  observe 
mortar  fires  ISO  TF  assault.  Phase  III  ends  with  enemy  forces  destroyed  in  OBJ 
BOSTON  and  TM  D  prepared  to  conduct  FPOL  through  TM  B  toward  OBJ  DALLAS 
(COA  A). 

•  COA  B  -  TM  D  follows  and  supports  TM  C  on  OBJ  CHICAGO,  and  prepares  to 
conduct  FPOL  through  TM  C  toward  OBJ  DALLAS. 

•  COA  C  -  TM  D  follows  and  supports  TM  A  on  OBJ  ATLANTA,  and  prepares  to 
conduct  FPOL  through  TM  A  toward  OBJ  DALLAS. 

(4)  PHASE  IV  (FPOL  to  OBJ  DALLAS)  -  This  phase  begins  once 
conditions  have  been  set  to  enable  TM  D  to  assault  remaining  enemy  forces  vie  OBJ 
DALLAS.  TM  D  FPOLs  TM  B  and  assaults  OBJ  DALLAS  to  complete  the  destruction 
of  enemy  forces  within  OBJ  KILLER  (COA  A).  Simultaneously,  TF  Mortars  cease  fire 
on  TGT  DF0104  once  TM  D  FPOLs  TM  B.  Sequentially,  TF  Mortars  cross  the  enemy 
obstacle  belt  through  the  lane  located  in  AXIS  C  and  establishes  an  MFP  vie  CP  3.  TF 
Scouts  remain  at  OP  1  and  provide  early  warning  IOT  protect  TF  north  flank.  This  phase 
ends  once  all  enemy  elements  are  destroyed  vie  OBJ  KILLER. 

•  COA  B  -TM  D  FPOLs  TM  C  and  assaults  OBJ  DALLAS  to  complete  the 
destruction  of  enemy  forces  on  OBJ  KILLER. 

•  COA  C  -TM  D  FPOLs  TM  A  and  assaults  OBJ  DALLAS  to  complete  the 
destruction  of  enemy  forces  on  OBJ  KILLER. 

(5)  PHASE  V  (SCREEN)  -  This  phase  begins  once  all  enemy  forces  are 
destroyed  within  OBJ  KILLER.  TF  OUTLAW  establishes  a  screen  arrayed  along  PL 
JEFFERSON.  TM  A  will  establish  the  screen  to  the  north,  TM  B  establishes  the  screen 
in  the  center,  and  TM  C  establishes  the  screen  in  the  south.  TM  D  consolidates  on  OBJ 
DALLAS  as  the  TF  reserve.  TF  Scouts  remain  at  OP  1  and  provide  early  warning  IOT 
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protect  TF  north  flank.  TF  Mortars  remain  at  CP  3  and  prepare  to  provide  indirect  fires 
ISO  TF  screening  operations. 


d.  Concept  of  Fires:  TF  Mortars  will  remain  under  TF  control  for  the 
duration  of  the  operation.  The  purpose  of  fires  for  this  operation  is  to  provide 
suppressive  fires  on  OBJ  KILLER.  Mortar  fires  initially  support  TF  breaching 
operations,  then  support  TM  D’s  assault  on  OBJ  DALLAS. 


e.  Coordinating  Instructions. 

•  Information  Requirements  [See  Appendix  2  (Commander’s  Critical  Infonnation 
Requirements)  to  Annex  B  (Intelligence)]. 

•  DP  Criteria  [See  Appendix  3  (Decision  Support  Matrix)  to  Annex  C 
(Operations)]: 

o  First  TM  to  establish  breach  determines  DP  1  criteria.  DP  1  is  located  in 
Phase  II  of  the  operation.  DP  1  drives  the  COA  to  be  executed  at  the  end 
of  Phase  II. 

4.  SERVICE  SUPPORT.  Current  /  Projected  CO/TM  &  Specialty  PLT  level 
combat  resource  status  [See  Appendix  1  (Resource  Status  Matrix)  to  Annex  I  (Logistics). 

5 .  COMMAND  and  SIGNAL.  (No  Change) 


SHATTUCK 

LTC 


OFFICIAL: 

HALL 

S3 

ANNEXES: 

ANNEX  B  (Intelligence) 

APPENDIX  1  (Enemy  SITEMP) 
APPENDIX  2  (Information  Requirements) 
ANNEX  C  (Operations) 

APPENDIX  1  (COA  Concept  Sketches) 
APPENDIX  2  (Execution  Matrix) 
APPENDIX  3  (Decision  Support  Matrix) 
ANNEX  I  (Logistics) 

APPENDIX  1  (Resource  Status  Matrix) 


122 


Appendix  1  (SITEMP)  to  Annex  8  (Intelligence)  to  OPORD  08-43  (Operation  Typhoon) 

MLCOA 


App^oiiiv  1  (RITFMP)  trt  Annuv  R  (In  rtPH PH  nJI-A.I  {DfWkratktn  Typhnnn) 

MDCOA 
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Appendix  2  (CCIR)  to  Annex  B  (Intelligence)  to  OPORD  08-43  (Operation  Typhoon) 
Commander’s  Critical  Information  Requirements  (CCIR) 


Priority  Intelligence  Requirements  (PIR): 


1 .  Are  T-72  tanks  present  vie  OBJ  KILLER? 

2.  What  is  the  enemy’s  remaining  combat  power  for  both  T-72s  &  BMPs  (alive  + 
templated)  once  TM  D  reaches  ABF  D? 

Friendly  Force  Information  Requirements  (FFIR): 

1 .  What  is  the  friendly  to  enemy  force  ratio  (ex.  3:1)  once  TM  D  reaches  PF 
DAMAGE? 

2.  What  are  the  Mortar  platoon’s  resource  statuses  once  they  reach  PF  DAMAGE? 
(Report  color  status  for  ammo,  fuel,  and  vehicles) 


Appendix  1  (Concept  Sketch)  to  Annex  C  (Operations)  to  OPOttO  08-43  (Operation  Typhoon) 


PL  JEFFERSON 


COA  A 

PL  RAMPAGE  PL  DAMAGE  PL  PHOE  NIX 


T  M  D  Assaults  in 
AXIS  B 


PL  RAMPAGE 


PL  0 AMA06 


PL  PHOENIX 
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COAB 


PLJEXIERSOY 


PL  RAMPAGE  PL  DAMAGE  PL  PHOENIX 


TM  D  Assaults  in 
AXIS  C 


9 


*TK 


PI  RAMPAGE 


PL  DAMAGE 


PL  PHOENIX 


PL  JEFFERSON 


COAC 

PL  RAMPAGE  PL  DAMAOE 
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Appendix  2  (Exec  jtion  Matrix)  to  Annex  C  (Operations)  to  OPORD  0843  [Operation  Typhoon) 
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Appenttx  3  (Decision  Support  Matrix)  lo  Annex  C  (Operations)  lo  OPORD  08-43  (Operation  Typhoon) 


Decision  Point  1  Criteria  (PI  RAMPAGE) 


COA 

Enemy  Conditions 

Friendly  Conditions 

Friendly  Action 

Location 

A 

Initial  hrparh  thru  onnmy 

obstacles  occurs  in  Axis  B 

TM  B  west  Of  PL  RAMPAGE 

TM  D  accault*  toward  OR  1  XII  1  FR 

in  AXIS  B  (Center  Zone) 

AxisB 

B 

Initial  breach  thru  enemy 
obstacles  occurs  In  Axis  C 

TM  C  west  Of  PL  RAMPAGE 

TM  D  assaults  toward  OBJ  KILLER 
In  AXIS  C  (South  Zone) 

AxisC 

C 

Initial  breach  thru  enemy 

obstacles  occurs  in  Axis  A 

TM  A  west  Of  PL  RAMPAGE 

TM  0  assaults  toward  OBJ  KILLER 

in  AXIS  A  (North  Zone) 

Axis  A 
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Appendix  1  (Reiource  Sttlui  Matrix)  to  Annex  I  (Logistic*)  lo  OPORO  0843  (Operation  Typhoon) 


Authorized  Unit  Basic  Load  for  Resupply  Operations: 


TF  Resource  UBL  (100%) 


Type 

QTY 

T/T 

1248 

25 

21600 

Fuel  (Gal) 

14676 

BFV 

24 

Tank 

24 
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APPENDIX  B.  RAID  SCENARIO  OPERATIONS  ORDER 


OPERATION  ORDER  08-44  (OPERATION  WHIPLASH) 


References:  Map,  Falcon  View  version  6,  1998,  A1  Icia  Maria,  Isconderia,  Special 


Time  Zone  Used  Throughout  Order:  Local 


Task  Organization: 


TMA(SE) 

TM  B  (SE) 

TM  C  (SE) 

1/A/Tank  PIT  (4  x  Ml) 

1/B/Tank  PLT  (4  x  Ml) 

1 /C/Mach  IN  PLT  (4  x  BFV) 

2/A/Tank  PLT  (4  x  Ml) 

2/8/Tank  PLT  (4  x  Ml) 

2/C/Mech  IN  PLT  (4  x  BFV) 

3/C/Meeh  IN  PLT  (4  x  BFV) 

3/8/Tank  PLT  (4  x  Ml) 

3/A/Tank  PIT  (4  x  Ml) 

TM  D  (ME) 

TF  Mortar* 

TF  Scout* 

1/D/Mech  IN  PLT  (4  x  BFV) 

A/Sec  (2  x  120mm) 

A/Wheeled  Sac  (2  x  HMMWV) 

2/D/Mech  IN  PLT  (4  x  BFV) 
3/D/Mech  IN  PLT  (4  x  BFV) 

8/Sec  (2  x  120mm) 

B/Wheeted  Sec  (2  x  HMMWV) 

1.  SITUATION. 

a.  Battlefield  conditions. 

(1)  Weather.  No  Change. 

(2)  Light  Data.  No  Change. 

(3)  Terrain.  AL  ICIA  MARIA  is  complex  urban  terrain  severely 
restricts  friendly  vehicular  movement.  Most  roads  facilitating  armor  vehicles  only  enable 
west-east  travel  in  column  formations.  Narrow  roads  surrounded  by  structures  create 
numerous  choke  points  and  kill  zones. 


(4)  Obstacles.  Enemy  is  anticipated  to  employ  IEDs  within  choke  points 
and  street  intersections. 


b.  Enemy  Forces.  AL  ICIA  MARIA  is  located  three  kilometers  west  of  the 
SCIRIAN  border,  and  has  become  a  major  point  of  infiltration  for  foreign  supplied 
weapons,  fighters,  and  money  being  tunneled  to  the  Anti-Isconderian  Forces  (AIF). 
HUMINT  sources  suggest  that  ABU  X  is  directly  overseeing  all  AIF  activities  in  and 
around  AL  ICIA  MARIA.  ABU  X  is  currently  listed  as  Number  4  on  the  Coalition  Force 
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(CF)  High  Value  Individual  (HVI)  target  list.  HUMINT  sources  revealed  that  ABU  X 
has  been  routinely  traveling  from  SCIRIA  to  AL  ICIA  MARIA  in  a  black  two  door  Opal 
for  the  past  couple  of  months.  HUMINT  sources  also  revealed  that  ABU  X  often 
conducts  meetings  from  safe  houses  located  vie  OBJs  DYLAN  and  BRUCE.  Recent 
SIGINT  indicates  that  ABU  X  is  currently  located  within  AL  ICIA  MARIA  and  plans  to 
meet  with  other  insurgent  leaders.  Additionally,  Isconderian  Security  Forces  (ISF)  report 
that  the  AIF  is  using  a  school  located  vie  OBJ  CRAZY  as  an  insurgent  recruiting  station. 
The  AIF  is  also  believed  to  have  established  an  improvised  explosive  device  (IED)  and 
vehicle-home  improvised  explosive  device  (VBIED)  factory  vie  OBJ  ADAM. 
Intelligence  sources  currently  estimate  20-30  AIF  personnel  operating  within  AL  ICIA 
MARIA. 

(1).  MLCOA  in  TF  OUTLAW  AOR  [See  Appendix  1  (SITEMP)  to  Annex  B 
(Intelligence)].  ABU  X  will  depart  AL  ICIA  MARIA  in  anticipation  of  CF  operations. 
Hard  core  fighters  will  conduct  attacks  consisting  of  IEDs,  anti-annor  ambush  teams  with 
multiple  RPGs,  and  limited  mortar  fires  to  discourage  CF  from  entering  into  key  AIF 
areas  of  operation. 

(2) .  MDCOA  in  TF  OUTLAW  AOR.  AIF  employs  VBIEDs  to  enable 
ABU  X  to  exfiltrate  east  toward  SCIRIA. 

(3) .  Enemy  Composition  (Templated). 

I 

HVI  *4  (Abu  X) 

J _ _ 


3  x  SVBIED  (MDCOA 
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c.  Friendly  Situation.  The  current  mission  is  part  of  ongoing  offensive 
operations.  Due  to  the  OPTEMPO,  logistical  are  experiencing  difficulties  with 
resupplying  forward  units.  Thus,  TF  OUTLAW  will  execute  this  mission  with  severely 
reduced  resources. 


2.  MISSION,  NLT  xxxxJUNxx,  TF  OUTLAW  raids  insurgent  support 
zones  in  AL  ICIA  MARIA  IOT  disrupt  AIF  operations  within  AO  OUTLAW, 

3.  EXECUTION. 

a.  OUTLAW  6  Intent: 

Purpose:  Disrupt  AIF  activities  within  AL  ICIA  MARIA. 

Key  Tasks: 

•  Conduct  precision  raids  against  specified  objectives  within  AL  ICIA  MARIA. 

•  Capture/Kill  HVI  #  4. 

•  Destroy  insurgent  safe  havens,  training  facilities,  and  munitions  factory. 

Endstate: 

•  Friendly:  TF  OUTLAW  preparing  for  future  COIN  operations  in  AOR 
OUTLAW. 

•  Enemy:  Insurgent  Groups  neutralized  and  unable  to  support  ongoing  AIF 
activities. 

b.  Concept  of  operations  [See  Appendix  1  (Concept  Sketch)  to  Annex  C 
(Operations)].  The  decisive  point  of  this  operation  is  the  capturing/killing  of  HVI  #  4 
through  rapid  and  violent  execution.  This  is  a  4  phase  operation:  (1)  Initial  assault;  (2), 
raids;  (3)  complete  AIF  destruction;  (4)  exfiltration.  Three  courses  of  action  (COA)  have 
been  planned  based  on  HVI  #  4’s  location.  These  conditions  will  drive  which  COA  is 
implemented  at  DP  1  [See  Appendix  3  (Decision  Support  Matrix)  to  Annex  C 
(Operations)]: 


•  COA  A  (initial  COA  to  be  executed)  -  HVI  #  4  located  vie  OBJ  DYLAN.  TM  D 
executes  raid  on  OBJ  DYLAN  and  captures/kills  HVI  #  4. 

•  COA  B  -  HVI  #  4  not  identified  within  AL  ICIA  MARIA.  TM  C  executes  raid 
on  OBJ  BRUCE. 

•  COA  C  -  HVI  #  4  located  vie  OBJ  BRUCE.  TM  A  executes  raid  on  OBJ 
BRUCE. 
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c.  Scheme  of  Maneuver  [See  Appendix  2  (Execution  Matrix)  to  Annex  C 
(Operations)]. 


(1)  PHASE  I  (INITIAL  ASSAULT)  -  This  phase  begins  with  TF 
Scouts  at  OP  1  to  gain  observation  on  OBJs  DYLAN  and  BRUCE.  Simultaneously,  TM 
C  establishes  SBF  Cl,  and  TF  Mortars  establish  MFP  vie  CP  1  to  enable  TF  freedom  of 
maneuver  during  the  initial  assault  into  the  town.  Sequentially,  TMs  A  &  D  attack 
toward  PL  TIGRIS  along  RTEs  AGGIES  and  DYNAMITE  respectively.  TM  B 
establishes  ATK  POS  B  and  prepares  to  attack  along  RTE  BONFIRE.  TF  Mortars  fire 
TGT  GRP  CJ7  once  TM  D  executes  PL  RHINE  to  suppress  possible  anti-armor  ambush 
teams  operating  in  the  northern  forest.  TM  C  departs  SBF  Cl  and  attacks  along  RTE 
CROW  to  PL  AMAZON  once  TM  A  executes  PL  AMAZON.  Simultaneously,  TM  B 
departs  ATK  POS  B  and  clears  RTE  BONFIRE  east  from  PL  RHINE  to  PL  AMAZON 
once  TMs  A  &  D  execute  PL  AMAZON.  This  phase  ends  with  TF  OUTLAW 
maneuvering  toward  specified  OBJs. 

(2)  PHASE  II  (RAIDS)  -  This  phase  begins  with  TM  A  executing  a 
raid  on  OBJ  ADAM  to  destroy  enemy  IED/VBIED  factory,  while  TM  B  clears  RTE 
BONFIRE  from  PL  AMAZON  to  PL  TIGRIS.  Sequentially,  TF  Mortars  fire  TGT 
DF0104  once  TM  D  executes  PL  TIGRIS  to  deny  enemy  exfiltration  east  from  OBJ 
DYLAN.  Once  TM  A  seizes  OBJ  ADAM,  TM  C  raids  OBJ  CRAZY  to  destroy  AIF 
recruiting  center,  while  TM  D  raids  OBJ  DYLAN  to  capture/kill  HVI  #  4  (COA  A).  TF 
Scouts  remain  at  OP  1  and  continue  to  observe  assigned  areas  of  observation.  This  phase 
ends  with  TMs  A,  C,  &  D  completing  raids  on  their  assigned  OBJs,  and  with  TM  B 
clearing  RTE  BONFIRE  east  to  PL  TIGRIS. 

(3)  PHASE  III  (COMPLETE  AIF  DESTRUCTION)  -  (DP  1  located 
in  this  phase)  This  phase  begins  with  TM  A  establishing  SBF  A1  to  suppress  enemy 
elements  located  vie  OBJ  BRUCE.  Sequentially,  once  TM  A  establishes  SBF  Al,  TMs 
C  &  D  complete  actions  on  their  OBJs  and  establish  SBFs  C2  &  D1  respectively  to 
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suppress  enemy  elements  vie  OBJ  BRUCE.  TF  Mortars  fire  TGT  DF0105  once  TM  D 
establishes  SBF  D1  to  suppress  enemy  elements  on  OBJ  BRUCE.  Once  all  SBFs  are 
established,  TM  B  executes  PE  TIGRIS  and  attacks  toward  OBJ  BRUCE.  TF  Mortars 
cease  fires  on  TGT  DF0105  once  TM  B  executes  PL  NILE.  Sequentially,  TM  B  raids 
OBJ  BRUCE  and  completes  the  disruption  of  AIF  activities  within  AL  ICIA  MARIA 
(COA  A).  TF  Scouts  remain  at  OP  1  and  continue  to  observe  assigned  areas  of 
observation.  This  phase  ends  with  ah  raids  complete  and  HVI  #  4  captured,  killed,  or 
confirmed  not  present  within  AL  ICIA  MARIA. 


•  Alternate  COAs  at  DP  1  -  In  the  event  HVI  #  4  is  not  located  vie  OBJ  DYLAN, 
TF  OUTLAW  prepares  to  execute  alternate  COAs  IAW  DP  1  criteria. 


o  COA  B  is  in  the  event  HVI  #4’s  location  cannot  be  identified  within  AL 
ICIA  MARIA.  During  COA  B,  TM  C  bypasses  SBF  C2  and  raids  OBJ 
BRUCE  to  complete  the  disruption  of  AIF  activities  within  AL  ICIA 
MARIA.  TM  B  executes  PL  TIGRIS  and  establishes  SBF  C2.  TMs  A  & 
D  remain  at  SBFs  A1  &  D1  respectively,  and  continue  to  suppress  OBJ 
BRUCE  ISO  TM  C.  TF  Mortars  cease  TGT  DF0105  once  TM  C  reaches 
SBF  C2. 


o  COA  C  is  executed  if  HVI  #  4  is  located  in  the  vicinity  of  OBJ  BRUCE. 
During  COA  C,  TM  A  immediately  departs/bypasses  SBF  A1  and  attacks 
toward  OBJ  BRUCE  to  capture/kill  HVI  #  4  IOT  deny  his  escape  from  the 
battlespace.  TMs  C  &  D  remain  at  SBFs  C2  &  D1  respectively,  and 
continue  to  suppress  OBJ  BRUCE  ISO  TM  A.  TM  B  holds  at  PL  TIGRIS 
along  RTE  BONFIRE.  TF  Mortars  cease  fire  on  TGT  DF0105  once  TM 
A  departs  SBF  Al. 


(4)  PHASE  IV  (EXFILTRATION)  -  This  phase  begins  on  order  (O/O) 
once  ah  OBJs  have  been  thoroughly  searched  and  ah  detainees  have  been  secured. 


d.  Concept  of  Fires:  TF  Mortars  will  remain  under  TF  control  for  the 
duration  of  the  operation.  The  purpose  of  fires  for  this  operation  is  to  enable  the  TF  to 
maintain  freedom  of  maneuver  during  the  duration  of  the  operation  by  providing 
suppressive  fires  on  pre-designated  targets.  Mortar  fires  initially  support  TM  D  as  they 
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maneuver  along  RTE  DYNAMITE,  then  during  TM  D’s  raid  on  OBJ  DYLAN.  Mortar 
fires  sequentially  support  the  raid  on  OBJ  BRUCE, 
e.  Coordinating  Instructions. 

•  Information  Requirements  [See  Appendix  2  (Commander’s  Critical 

Infonnation  Requirements)  to  Annex  B  (Intelligence)]. 

•  DP  Criteria  [See  Appendix  3  (Decision  Support  Matrix)  to  Annex 
C  (Operations)]: 

o  HVI  #  4’s  location  detennines  DP  1  criteria.  DP  1  is 
located  in  the  beginning  of  Phase  III  of  the  operation.  DP  1  drives  the 
CO  A  to  be  executed  during  Phase  III. 

4.  SERVICE  SUPPORT.  Current  /  Projected  CO/TM  &  Specialty  PLT  level 
combat  resource  status  [See  Appendix  1  (Resource  Status  Matrix)  to  Annex  I  (Logistics). 

5.  COMMAND  and  SIGNAL.  (No  Change) 

SHATTUCK 

LTC 


OFFICIAL: 

HALL 

S3 

ANNEXES: 

ANNEX  B  (Intelligence) 

APPENDIX  1  (Enemy  SITEMP) 
APPENDIX  2  (Infonnation  Requirements) 
ANNEX  C  (Operations) 

APPENDIX  1  (COA  Concept  Sketches) 
APPENDIX  2  (Execution  Matrix) 
APPENDIX  3  (Decision  Support  Matrix) 
ANNEX  I  (Logistics) 

APPENDIX  1  (Resource  Status  Matrix) 


134 


Appendix  1  (SITEMP)  to  Annex  B  (Intelligence)  to  OPOR  D  08-44  (Operator*  Whiplash} 

MLCOA 


Appendix  1  (SITEMP)  to  Annex  B  (Intelligence)  to  OPOR  0  08-44  (Operation  Whiplash) 

MDCOA 
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Appendix  2  (CCIR)  to  Annex  B  (Intelligence)  to  OPORD  08-44  (Operation  Whiplash) 


Commander’s  Critical  Information  Requirements  (CCIR) 


Priority  Intelligence  Requirements  (PIR): 


3.  Is  the  enemy  employing  VBIEDs  within  AL  ICIA  MARIA? 

4.  What  is  the  enemy’s  remaining  combat  power  for  both  RPG  Teams  and  VBIEDs 
(alive  +  templated)  once  TM  C  reaches  PL  TIGRIS? 

Friendly  Force  Information  Requirements  (FFIR): 


3.  What  are  TM  A’s  resource  statuses  once  they  reach  PL  AMAZON?  (Report  color 
status  for  ammo,  fuel,  and  vehicles) 

4.  What  is  the  friendly  to  enemy  force  ratio  (ex.  3:1)  once  TM  B  reaches  PL 
AMAZON? 

Append*  1  (Concept  Sketch)  to  Annexe  (Operation*)  to  OPORD  08-c4  (Operation  Whiplash) 
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Appendix  £  (Execution  Matrix)  to  Annex  C  (Operations)  to  OP  OR  D  Ofl  44  (Ope  ration  Whip  (ash) 
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_ Appendix  3  IDecisioci  Support  Matrix)  lo  Annex  C  (Operations)  to  OPORD  03  44  (Opera t>on  Whiplash) 

Decision  Point  1  Criteria  (PI  NIIE)  


COA 

Enemy  Conditions 

Friendly  Conditions 

Friendly  Action 

Location 

A 

HVI » 4  location  confirmed  at 

OBJ  DYLAN 

TM  D  maneuvering  to 
establish  SBF  D1 

TM  B  raids  OBJ  BRUCE 

OBJ  BRUCE 

TM  C  man  ueve  ring  to 
establish  SBF  C2 

TM  A  at  SBF  A1 

B 

HVI  if  4  not  located  within  AL 

ICIA  MARIA 

TM  0  man  euverlng  to 
establish  SBF  DI 

TM  C  raids  OBJ  BRUCE 

OBJ  BRUCE 

TM  B  man  euve  ring  to 
establish  5BFC2 

TM  A  at  SBF  A1 

C 

HVI  >1 4  located  vie  OBJ 

BRUCE 

TM  D  man  euverlng  to 
establish  SBF  Dl 

TM  A  raids  OBJ  BRUCE 

OBJ  BRUCE 

TMC  man  ueve  ring  to 

establish  SBF  C  2 

TM  B  hold  s  at  PL  TIGRIS 
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Appendix  1  (Resource  Statu*  M»lri»)  lo  Annex  I  (loqutio)  to  OPORO  08-44  (Operation  Whiplash) 


T ask  Force  Level  Resource  Status 


Company  Level  Resource 
Status 


Platoon  Level  Resource 
Status 


Authorised  Unit  Basic  Load  lor  Resupply  Operations: 


TF  Resource  UBL  (100%) 

Type 

QTY 

TIT 

1248 

25 

21600 

Fuel  (Gal) 

14676 

BFV 

24 

Tank 

24 
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APPENDIX  C.  PARTICIPANT  DEMOGRAPHIC  SURVEY 


1.  Age  _ 

ZOmk  M  F 

1  Ra  i* /Grade  _ 

4.  Caiwr  Efeandt  (Bl  Amo): _ 

EL  Ymr  of  Samoa:  _ 

pound  to  neanat  yw) 

B.  An  you  color  bM?  Y  N 

7.  Da  you  hno  comrand  aserienca? 

Y  N 


If  Too.  total  comrandinc  _ 

•  *  -i  ^ 

(nwntwj 

B.  Have  you  aver  been  eedaiedaeaSetle 

CqihfeiF  Y  N 

ir  Yio,  total  BaUo  Captabi  fen  a: _ 

frnonthg 


91  Hava  you  ovar  baan  oaripied  aa  abhor  d 

tha  fotmtag?  (marie  al  hot  apply) 

no.  of  monow  __ 
_ Aoriot3-3  No.  of  months _ 


10.  Do  you  hove  combat  experience? 

Y  N 

irYae, 

Nunbar  of  Combat  Daptaymantc  _ 

Total  Combat  Tkne:  _____ 

(rrtonthf 


11.  Do  you  hove  any  opvdfcno  other  fan  war 
(MOOTW)  dap  byrrant  experience?  Y  N 


Number  of  MOOTWdaptoynwntoc  _ 

TotalMDOYWOapioyimnt  Tima  _____ 

frnonthj 

120o  you  hova  any  Coerbet  TraMnOontor 
(CTC)  rotation  experience  aa  a  ptayarP 

Y  N 

IfYaa, 


Total  nunbar  of  CTC  ptayar  ratafemc 


13.  Oo  you  have  experience  who  FBCB2  or 
BFTP  Y  N 

If  Yaa.  typa  of  FBC82  experience?  Omaikall 

titat  nr) 


Trameo 


Trataar 


14.  Haw  you  ewrboanaeafanodoaa 
"Pudotar  during  arimbbacarial 

_ ■  _  _w  a#  aa 

OMfOBBr  T  n 


IfYaa, 

MpiN  rata  aa  a  Pudntan 


CBlCoCA) 


16.  Do  you  ptay  htanctba  video  go  mao  on  a 
computer  or  oome  other  ganfeig  oyatantf 

IfYaa, 

Arango  houra  par  weak  opart  gamtag: _ 


16.  Rata  jpot^r  confdenca  l^arel  ua^ng  a 
computar  bobvaanl  (rary  km)  to  6  (wry 


141 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


142 


APPENDIX  D.  RAPTOR  POST-TRAINING  TEST 


Directions:  The  following  questions  pertain  to  specific  functions,  tools,  options, 
displays,  and  representations  presented  by  the  RAPTOR  interface.  The  purpose  of  this 
questionnaire  is  to  ensure  proper  levels  of  knowledge  required  for  the  successful 
operation  of  the  interface  are  achieved  prior  to  the  conduct  of  the  experimental  trials. 
Please  read  each  question  carefully,  then  circle  the  letter  that  corresponds  to  the 
correct  answer. 

1 .  Alternate  courses  of  action  (COA)  can  be  previewed  by: 

A.  Clicking  and  dragging  synchronization  points  in  the  map  display 

B.  Clicking  and  dragging  synchronization  points  in  the  synchronization  matrix 

C.  Placing  the  cursor  over  a  desired  COA  button  located  under  the  synchronization 
matrix 

D.  All  of  the  above 

2.  BN  level  resource  color  statuses  can  be  detennined  by: 

A.  Requesting  a  SITREP  from  subordinate  units 

B.  Referencing  the  individual  resource  bars  located  in  BN  level  resource  chart 

C.  Both  A  &  B 

D.  None  the  above 

3.  PLT  level  resource  color  statuses  can  be  determined  by: 

A.  Selecting  a  desired  PLT  bubble  in  the  Unit  Control  Tree,  then  referencing  the 
individual  resource  bars  located  in  PLT  level  resource  chart 

B.  Requesting  a  SITREP  from  subordinate  units 

C.  PLT  level  resources  statuses  cannot  be  determined 

D.  None  of  the  above 

4.  Alternate  COAs  can  be  executed  by: 

A.  Pointing  and  clicking  on  a  desired  COA  button,  then  pointing  and  clicking  on  the 
current  COA  selection  button 

B.  Clicking,  dragging,  and  releasing  icons  in  the  map  display 

C.  Clicking  and  dragging  the  control  slider  located  above  the  synchronization  matrix 
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D.  All  of  the  above 


5.  The  following  graphic  control  measure  represents: 


A.  Support  by  fire 

B.  Attack  by  fire 

C.  Breach 

D.  None  of  the  above 

6.  The  following  graphic  control  measure  represents: 

tl 

A.  Breach 

B.  Direction  of  attack 

C.  Blocking  position 

D.  None  of  the  above 

7.  The  following  graphic  control  measure  represents: 

y-c 

A.  Attack  by  fire 

B.  Breach 

C.  Support  by  fire 

D.  None  of  the  above 

8.  The  following  graphic  control  measure  represents: 
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A.  Coordination  point 

B.  Decision  Point 

C.  Check  point 

D.  None  of  the  above 

9.  The  enemy  resource  chart  provides  information  for: 

A.  Quantities  of  identified  (alive/known)  enemy  equipment 

B.  Quantities  of  destroyed  enemy  equipment 

C.  Quantities  of  templated  (anticipated)  enemy  equipment 

D.  All  of  the  above 

10.  Follow-on  /  next  tasks  to  be  executed  by  subordinate  units  can  be  anticipated  by 
referencing  text  cells  to  the  right  of  those  cells  currently  intersected  by  the  blue 
timeline  in  the  synchronization  matrix 

True  /  False  (Circle  One) 

1 1 .  Current  force  ratios  can  be  determined  by: 

A.  Calculating  the  number  of  remaining  enemy  and  friendly  vehicles 

B.  Referencing  where  the  reflecting  line  intersects  the  right  edge  of  a  display  grid  in 
the  force  ratio  display 

C.  Force  ratios  cannot  be  determined 

D.  None  of  the  above 

12.  The  below  icons  represent: 

m  □ 

A.  Mortar  System  /  Howitzer 

B .  T ank  /  Infantry  F ighting  V ehic  le 

C.  Anti-Tank  Rocket  Launcher  /  Building 

D.  None  of  the  above 
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13.  The  below  icons  represent: 


t 

6 


A.  Mortar  System  /  Anti-Tank  Rocket  Launcher  (RPG) 

B.  Howitzer  /  Anti-Aircraft  Gun 

C.  Tank  /  HMMWV 

D.  None  of  the  above 

14.  The  below  icon  with  a  red  diamond  background  represents: 

□ 

A.  Unknown  Wheeled  Vehicle 

B.  Infantry  Fighting  Vehicle 

C.  VBIED 

D.  None  of  the  above 
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APPENDIX  E.  BASELINE  POST-TRAINING  TEST 


Directions:  The  following  questions  pertain  to  specific  functions,  tools,  options,  menus, 
and  representations  presented  by  the  Baseline  interface.  The  purpose  of  this 
questionnaire  is  to  ensure  proper  levels  of  knowledge  required  for  the  successful 
operation  of  the  interface  are  achieved  prior  to  the  conduct  of  the  experimental  trials. 
Please  read  each  question  carefully,  then  circle  the  letter  that  corresponds  to  the 
correct  answer. 

1 .  Reports  can  be  accessed  by: 

A.  Selecting  the  Applications  button 

B.  Selecting  the  CMD  Directives  button 

C.  Selecting  the  FIPR  button 

D.  All  of  the  above 

2.  Types  and  quantities  of  destroyed  enemy  vehicles  can  be  detennined  by: 

A.  Accessing  SPOT  reports  in  the  FIPR  menu 

B.  Requesting  a  SITREP  from  subordinate  units 

C.  Accessing  enemy  battle  damage  assessment  reports  (E-BDA)  in  the  FIPR  menu 

D.  None  of  the  above 

3.  Subordinate  unit  resource  statuses  (e.g.,  fuel,  ammunition,  etc.)  can  be  detennined  by: 

A.  Referencing  unit  icon  color  codes  in  the  map  display 

B.  Accessing  logistical  reports  (LOGSTAT)  located  under  the  Routine  tab  in  the 
FIPR  menu 

C.  Requesting  a  SITREP  from  subordinate  units 

D.  All  the  above 

4.  TF  COAs  can  be  changed  by: 

A.  Clicking  a  desired  course  of  action  (COA)  radio  button  in  the  Command  Directives 
menu,  then  clicking  the  execute  button 

B.  Clicking,  dragging,  and  releasing  combat  resource  icons  in  the  map  display 

C.  Developing  and  sending  fragmentary  orders  (FRAGO)  in  the  FIPR  menu 

D.  All  of  the  above 
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5.  Approximate  the  TF  Level  fuel  status  by  percentage  and  color  convention  using  the 
following  data  (TF  full  authorized  UBL  for  Fuel  =  14,676  Gal): 


Formula: 


Sum  Total  of  Remaining  Resource 

Authorized  UBL 


TM  A  current  fuel  status  =  3,540  Gal 
TM  B  current  fuel  status  =  4,500  Gal 
TM  C  current  fuel  status  =  2,580  Gal 
TM  D  current  fuel  status  =  1,620  Gal 
Mortars  current  fuel  status  =  280  Gal 
Scouts  current  fuel  status  =  100  Gal 

A.  75%  /  Amber 

B.  68% /Red 

C.  86% /Green 

D.  None  of  the  above 

6.  The  following  graphic  control  measure  represents: 


A.  Support  by  fire 

B.  Attack  by  fire 

C.  Breach 

D.  None  of  the  above 

7.  The  following  graphic  control  measure  represents: 

U 

A.  Breach 

B.  Direction  of  attack 

C.  Blocking  position 

D.  None  of  the  above 
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8.  The  following  graphic  control  measure  represents: 


A.  Attack  by  fire 

B.  Breach 

C.  Support  by  fire 

D.  None  of  the  above 

9.  The  following  graphic  control  measure  represents: 


A.  Coordination  point 

B.  Decision  Point 

C.  Checkpoint 

D.  None  of  the  above 

10.  Calculate  the  friendly  to  enemy  force  ratio  (ex.  3:1)  using  the  following  data: 


Total  #  Alive  Friendly  Tanks  &  BFVs 
total  n  Alive  +  Anticipated  Additional  T-/7s  &  BMPs 


Formula - > 


Remaining  friendly  vehicle  strength  -  24  x  Tanks;  20  x  BFVs 
Remaining  known  /  alive  enemy  vehicle  strength  -lx  T-72;  6  x  BMPs 
Unidentified,  but  anticipated  (i.e.,  template)  additional  enemy  vehicles  - 
1  x  T-72s;  3  x  BMPs 

A.  5:1 

B.  2:1 

C.  4:1 

D.  None  of  the  above 

1 1 .  Current  TF  combat  power  (i.e.,  SLANT)  can  be  detennined  by: 

A.  Requesting  a  SITREP  from  subordinate  units 
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B.  Accessing  each  subordinate  unit’s  most  current  logistical  report  (LOGSTAT)  in 
the  FIPR  menu,  then  calculating  the  quantities  of  operational  vehicles  by  type 

C.  TF  combat  power  cannot  be  determined 

D.  None  of  the  above 

12.  The  below  icons  represent: 

m  □ 

A.  Mortar  System  /  Howitzer 

B.  Tank  /  Infantry  Fighting  Vehicle 

C.  Anti-Tank  Rocket  Launcher  /  Building 

D.  None  of  the  above 

13.  The  below  icons  represent: 


A.  Mortar  System  /  Anti-Tank  Rocket  Launcher  (RPG) 

B.  Howitzer  /  Anti-Aircraft  Gun 

C.  Tank  /  HMMWV 

D.  None  of  the  above 

14.  The  below  icon  with  red  diamond  background  represents: 

□ 

A.  Unknown  Wheeled  Vehicle 

B.  Infantry  Fighting  Vehicle 

C.  VBIED 

D.  None  of  the  above 
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APPENDIX  F.  RAPTOR  STUDY  FEEDBACK  SURVEY 


Directions:  The  following  statements  concern  your  perceptions  about  the 
different  displays,  options,  tools,  etc  provided  by  the  RAPTOR  interface.  Please  rate  the 
strength  of  your  agreement  for  each  statement  below  by  placing  a  check  mark  next  to  the 
applicable  number  on  the  scale.  Please  provide  any  additional  comments  that  will  assist 
researchers  in  determining  the  overall  ability  of  RAPTOR’S  interface  design  to 
effectively  assist  user’s  as  they  execute  the  C2  of  tactical  operations. 

1.  The  individual  resource  bar  chart  color  codes  used  in  the  Friendly  Resource  Display 
enables  rapid  comprehension  of  unit  combat  effectiveness: 


1 

r  2 

3 

r  4 

-  5 

Strongly  Disagree 

Disagree 

Neutral 

Agree 

Strongly  Agree 

2.  The  Force  Ratio  Display  facilitates  decision-making  by  enabling  users  to  quickly 
determine  which  force  (friendly  or  enemy)  has  a  superior  advantage: 


r  1 

<-  2 

r  3 

r  4 

r  5 

Strongly  Disagree 

Disagree 

Neutral 

Agree 

Strongly  Agree 

3.  The  Control  Tree  enables  users  to  quickly  determine  friendly  resource  statuses  at  finer 
or  courser  levels  of  detail  (e.g.,  Platoon  level  status  vs.  Battalion  level  status): 


r  1 

r  2 

r  3 

r  4 

5 

Strongly  Disagree 

Disagree 

Neutral 

Agree 

Strongly  Agree 
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4.  The  course  of  action  (COA)  buttons  assists  with  decision-making  by  enabling  users  to 
rapidly  access  and  view  alternate  actions  friendly  forces  can  execute  if  required: 


r  i  r  2 

r  3 

r  4 

'  5 

Strongly  Disagree  Disagree 

Neutral 

Agree 

Strongly  Agree 

5.  Information  provided  in  the  Synchronization  Matrix  enables  users 

friendly  force  activities  by  time,  phase,  and  event: 

to  anticipate  future 

'1  '*2 

'  3 

r  4 

'  5 

Strongly  Disagree  Disagree 

Neutrai 

Agree 

Strongly  Agree 

6.  The  Enemy  Resource  Chart  reduces  uncertainty  by  enabling  users  to  quickly  determine 
enemy  strength  and  combat  effectiveness: 


r  1 

r  2 

r  3 

r  4 

5 

Strongly  Disagree 

Disagree 

Neutrai 

Agree 

Strongly  Agree 

7.  Other  observations  concerning  the  effectiveness  or  usefulness  of  any  of  the  interface 
displays,  tools,  options,  etc.  that  enable,  assist,  or  impede  the  user’s  ability  to  conduct  C2: 
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